Features such as syntax highlighting, red squiggly underlines and autocompletion require that you lexically and semantically parse the source code. Lexical parsing means reading the raw text stream and breaking it up into a stream of tokens; a semantic parser then reads those tokens and figures out what they mean in context, emitting a code DOM (called an Abstract Syntax Tree). The final step is to resolve the nodes in the AST into .NET types and members.
Much of the heavy lifting is done by NRefactory and SharpDevelop.Dom, written by Daniel Grunwald, Andrea Paatz, and Mike Krüger (as part of the SharpDevelop project). These are fantastic libraries: writing a fully-featured lexical and semantic language parser in itself is no easy task; writing a well-structured type resolution DOM that can correctly handle the nuances of C# 3.0 type inference is a major achievement.
Because LINQPad ships as a single click-and-run executable, all the libraries are included as embedded resources. (Look at the last example on this page to see how). A bonus of this approach is that you can compress the libraries at the same time, reducing the executable size.
LINQPad automatically patches itself by downloading updates into its Application Data folder. It then checks that the new assembly has the same strong name, and if so, forwards to that executable, which then writes itself back to the original file. This may fail (depending on the permissions of the folder to which LINQPad was downloaded) so LINQPad always checks whether there's a later signed version of itself in its Application Data folder when started.