This chapter includes two recipes. These two recipes demonstrate how to examine two very important request items for bots:

• Cookies
• Forms

    Cookies and forms are used by many websites. This book has an entire chapter devoted to each. Chapter 7, “Responding to Forms” discusses HTML forms. Chapter 8, “Handling Sessions and Cookies” discusses cookies. For now how to examine cookies in a request will be explained.

Recipe #2.1: Examining Cookies

    Cookies are used to maintain a state in a web server. A web server can attach a cookie to a response so that it can identify that browser when the web server sees another request from this web browser. Cookies will be discussed in much greater detail in Chapter 8, “Handling Sessions and Cookies”. For now we will simply examine a cookie in the network analyzer.

    To see cookies in action, visit a web site that makes use of cookies. The following page, on the HTTP Recipes site, uses cookies:

http://www.httprecipes.com/1/2/cookies.php

    The contents of this page are shown in Figure 2.12.

Figure 2.12: Ready to Create a Cookie

    When the page is first accessed, there will be no cookie, so the cookie will show the value of “Undefined”. When the “Set Cookie” button is clicked, the cookie’s value can be set to any value.

    Cookies always have a name. This cookie is named test-cookie. Remember this name! It will allow you to locate the correct packet in the network monitor.

    Before clicking anything, start WireShark. If the cookie was already set, ensure you click “Clear Cookie” before continuing. Begin capturing packets and return to the web browser. Once back at the web browser, select “Set Cookie”. Enter a value for the cookie, such as “Hello”, and you will be taken back to the page shown in Figure 2.12. However, this time, the value previously set to the cookie to should be displayed.

    Select WireShark and stop capturing packets. Filter to just HTTP packets in WireShark and look for the HTTP response just after the POST 1/2/cookies-set.php request. Figure 2.13 shows this.

Figure 2.13: Cookie as Part of a HTTP Response

    Notice the cookie? It is created by the Set-Cookie HTTP tag. Once the server has set a cookie, the browser must echo this cookie with each request. Look at the next request, which is GET /1/2/cookies.php. This request can be seen in Figure 2.14.

Figure 2.14: Cookie as Part of a HTTP Request

    Notice the Cookie header in the request above. This will be sent by the web browser now that the server has requested it. This tag now allows the server to identify this particular web browser.

    Tracking cookie usage can be very important when writing a bot. Using a network analyzer can assist in seeing how a web server is making use of cookies.

Recipe #2.2: Examining Forms

    Forms are another key element of most web sites. Using the network analyzer, it can quickly be determined how a web server makes use of forms. Forms will be covered in much greater detail in Chapter 7. For now, capturing forms with a network analyzer will be covered.

    To demonstrate HTML forms, the following URL from the HTTP Recipes site will be used:

http://www.httprecipes.com/1/2/forms.php

    Figure 2.15 shows the contents of this URL.

Figure 2.15: An HTML Form

    This form allows a user name and password to be entered. Turn on packet capturing in WireShark to see what happens when this form is submitted. Fill in the correct user name and password and click the “Login” button. This should allow a login to the web site.

    Once logged in, stop capturing packets. Examine the HTTP request, labeled POST /1/2/forms2.php, this will reveal Figure 2.16.

Figure 2.16: An HTTP Form Request

    As seen in the above figure, the form data is communicated in the following line:

uid=guest&pwd=guest123

    This is the format in which a web browser sends form data back to the web server. When a bot is created to respond to forms, data must be sent in this format.