Summary

    This book is about how to write programs that browse the web, just as a human does. Such programs are called bots. To create bots, it is important to review how web browsing works at a technical level, to form a foundation for the rest of the book. In this chapter, the general structure of web browsing and HTTP requests were explored. The two recipes in this chapter were both web servers. The first web server did nothing more than displays the text “Hello World” to any request; however, the second example implemented a full file based web server. Both of these recipes focused on socket programming and web servers. Socket programming uses the Socket class provided by C#. Although socket programming is touched on, most of this book will focus on creating applications that access web servers. These applications will mainly use the C# HTTP classes to access a web server, rather than direct sockets.

    The next chapter will show how to use certain tools to examine the interaction between a web browser and server. These tools will be very helpful, when you create programs of your own to access web servers. They will allow the programmer to understand exactly what information the desired web server expects.