Sunday, August 16, 2009

How to record the behavior of Internet users

I would like to discuss about this interesting topic which is sometimes wrongly perceived by many people as a menacing way practiced by some, if not many, Internet companies to invade one's privacy. I supposed there is a huge gap or the level of understanding that separates the consumers and the business people.

The technological perspective that I am illustrating in this article is about implementing "click-tracking". For software engineers that really know and understand whatever they are implementing, the server has no control over the dynamic HTML pages once the bits and bytes are presented to the end-users. For now, do we agree unanimously at this stage?

Great, thank you for agreeing to my thoughts! If so, I shall elaborate further about how "click-tracking" is implemented, especially at the anchor element <a> in the HTML document. There are at least three significant ways to track the links the users are clicking. I am taking an online video company as an example.

[1] Redirecting users to different servlets
This is the most common way to track user clicks. Imagine you are a registered user and while your session is still active:
1. You do a keyword search
2. You get a compiled list of result items in the result page
(eg: http://www.example.com/search/?q=keyword_string&pg=1)
3. You click at the link you think is most relevant to you
(eg: http://www.example.com/view/?videoID=12345678)
4. You enjoy watching the selected video
5. You demand for more videos and you reiterate step 1 again.
But WAIT ... what laymen do not know is that while they are busy with their viewing activities, the servers are busy harvesting the clicks of the links. Technically speaking, the servlet that is related to the URL request will update the user database with the unique video IDs, which I think is the most important element. Many interesting things can take place from here and one of them is to count the number of viewed videos.

[2] AJAX
Some of you might have this question popping in your mind "But what if the page is loaded dynamically using AJAX?". The implementation is actually quite similar to the one mentioned previously but with a slightly different flavor. By using AJAX, you can implement an asynchronous method to pass the video ID as an argument to the servlet using GET or POST method. The servlet will then take care of the rest without the user knowing whatever happens behind the scene (or their clicks are monitored).

[3] Javascript / AJAX
The other method to perform "click-track" is by using Javascript. This is used in the scenario where the search result page contains links that redirect users to the original sources and not within their domain. In other words, a totally different URL altogether. A GET or POST method is used to send the data back to the server.

While the 3 suggested implementations seem similar, they are actually not the same. I hope you find this article a good read and insightful about performing "click-track".

2 comments:

  1. I guess you were missing another 4th way: linking an image hosted on your server. If you put such link to any webpage you tracking, it'll only become a matter of parsing http headers on your server that will allow you to track your victims.
    I think this is the way it was done in the early Internet days.

    ReplyDelete
  2. thank you for your comment, budchee! :D

    ReplyDelete