Work on messaging System goes on

I only worked on the server and database related stuff. Todays work mostly contained work on dealing with messages. This was actually much more work than I originally expected. Having to deal with message de and encoding to/from a sendable format, identfying users involved and dealing with messages to offline and HTTP hidden users. Only the fact that the recipient could be a Chatroom with a number of users makes the situation much more complicated. All in all plenty of work. I am still not sure how to deal with that log. My latest idea is to store it as textfiles to reduce load on the database, but here again I am not sure whether this would be faster at all and how to synchronize it. One way would be a Buffer which is being emptied into the Log after time intervals. Furthermore two log Files would be stored which work accross the servers.

• HTTP Log: Would be filled by the Socket server and read + emptied by the HTTP Server.
• SOCKET Log: Would be filled by the HTTP Server and read + emptied by the Socket Server.

The main problem is just how to read those while the other side is writing on them? Probably just wait a few millisekonds and try it again. But again, it all only seems to be good if it is faster than a Database. I will have to consult some Database experts probably ;-)

Database work is going on

I did not manage as much as I wanted, but it is still some good work considering that I forgott again plenty of SQL Stuff. Well, the server is running, the DSN connection works, my connection Pool as well and I am already able to verfiy the userlogin through the database. I will only do as much as necessary as the time schedule is too tight to mess around with optimisation. Currently I work on reading out the buddyList. Tomorrow will get on much faster with it after I have worked myself into it again. In addition to the database I also lost some time by meeting a Bug in the Java String.split(String) method till I figured out that the method is not able to deal with all characters for splitting. Hoping for a good day tomorrow.

Working on the database

I finally startet with some real database work. Till yet I only had a small Dummy Connection which was just enough to simulate a login and the buddyList transfer to the client. The last work on the clients side was to send a message.  This required to work a little more on the server so I decided to now stick with the server and start with the DB. During my work on the I had to add two components into the DB.

• A chat object with a userlist within the chat.
• A Database Log to allow communication between both servers.

I got the idea with the Database Log when I startet with messages. The easiest way wold be that each client would regularly poll the DB and to check whether anything has changed (new messages, new buddies,...). How am I planning to implement it? The design is not finished yet but I already got the main concept. Each time a buddy request, message or other user related event occurs, it will be logged into the Log Table. On the TCP or Socket based server one static class with a separate DB-Connection would query this table and notify all client handlers using MVC about those changes. In this case the entry would be delted from the log. In order to keep this method efficient, all entries older than  a minute will be deleted. At the moment I have to think about how to log the events and which events. Currently I am thinking of Storing it in the following Format: LogID, Time, User, Event The Event would be something very basic like "Received Message" and the notified user handler would check all received messages for this user within the db.

BuddyList exchange initially works

It was a productive day today. The client now receives the Buddylist when logging in into the server and the UI shows the list. I also modified the protocol design so, that I don't rely on responses in the protocol. I did so by not sending the buddylist as one piece putting away Avatardownloads from the TCP protocol.

• In more detail: The buddylist will now be sent as single buddies and the client adds them one by one, instead of sending the whole bunch at once (Java automatically split my one message into several).
• The Avatar transfer went through a more radical design change. I decided to use a separate Imageserver and the client will download the images from the Avatar/Image Server using HTTP (no mather which connection is being used).

At the moment I am working on the implementation of the Avatar exchange and on further Avatar handling. Automatically load a backup avatar if the avatar has not been downloaded yet. Also some MVC stuff in case the Avatar is ready or the user goes offline,..... Nevertheless, there is still a lot to do!

Protocol Implementation

After a short break, I am now busy again with the project. Unfortunately not anymore with the speed I am used to. The Buddy List implementation was much harder than I expected. Mostly MVC redundancy caused difficulties and I had to change some structure in the background. In order to check how it works I have to continue with the protocol which also gives me a headache. Here it’s the synchronisation which is kinda tricky, especially since the protocol has to be split into two lower level protocols. My current design involves the usage of message which requires time accurate responses(some message need a correct response from the other side). E.g.: • Server: Send a Buddy • Client: Say whether Avatar is required • Server: Send Avatar (if required) Due to the required asynchronous communication it could happen different messages come in while waiting for a response. At the moment I am planning to change the protocol design so, that I don’t depend on responses. I’m going to see tomorrow how I’m going to change it. Especially I have to check how to deal with file transfers.

Development has started

Over Christmas I used a few days to just sit on the implementation. What has been done till yet: Client:

• Main Window Prototype
• First Localisation system
• Settings Reader/Writer
• Icon Handler
• Basic Image Handler (Still needs methods to write images as jpeg or png)
• Connection Interfaces designed (very early alpha stage)
• Many Util classes developed (File Handler, User Container,...)

Server:

• First attempts to create TCP connection

I will continuew to work intensively for a few more days and then I will start prepairing for exams.

Next Step

So, I finally finished my interim report. No the next step is to start with some implementation work. I first want to start with the main protocol and then I will look further. For now I wish everyone a merry christmas!

Interim Report

Having serious problems with my Laptop last Weekend, I finally fixed it and am now able again to continue with my work. I requested a computer from DIT just in case my machine would brake down over Christmas. As this event interrupted my work I talked with my supervisor and we arranged a later submission-date for the interim report. Having done a decent amount of work for it, I will start with the implementation stage shortly.

Interim Report

Finally the Assignments are over and I can start with the interim report. Till now I mostly did some research in case of the protocol. I have a rough Idea of how to implement it now. The propably hardest part will be the implementation of file transfers, but I will see. This kind of stuff is being used regularly all over the web and should not be too complicated.

Protocol

After researching again a little into available XMPP APIs I finally decided to create my own Protocol.

The main Problem here by is the http connection. It was a little tough to find information persistent connection using http.
I had a talk with Michael Collins and he told me that it's not possible anyway as http is stateless. He recommended me to use standard Sockets. As a result I did some further research how XMPP managed to connect through http and most important: receive messages from the server.
On Wikipedia I found a solution. It uses / used two approaches. HTTP polling and HTTP binding. For HTTP binding i will have to make some more research.

Polling

Polling is very simple.It uses frequent GET and POST requests to the server and the server replies all messages which have been stored.

Binding

Binding is a little more complex. Wikipedia says, that it uses longer lived HTTP connections. I interpret it in a way, that the server sends out all messages but it won't close the connection. It could be compared with viewing a web site using a bad internet connection where the page is being loaded slowly line for line. Difficulties for this approach would appear in many ways. How tom implement it that the server will not create xxx bound connections to one client, as the connection is bound only one way. Another very important part is how to deal with time outs?

A Possible Solution

I also found out that XMPP also uses Standard Sockets - as recommended by Michael Collins. My idea is now to create a double implementation. Using standard Sockets and as an alternative HTTP if a socket connection could not be established. This would requires a good and clean implementation to avoid time loss due to unnecessary double implementation. Standard sockets for with full support how it is supposed to be and HTTP polling with a refresh rate between 1 and 20 seconds. This would slow decrease the response speed, but it would still allow a considerably good communication. As this is a quite large project I am considering HTTP binding only as an option if enough time is left in the end of the project as implementation would be unnecessary complex for the benefits in comparison to Polling. Nevertheless binding still gives enough potential should this project become a commercial success and a reduction of server load would become necessary.

File Transfer

For file transfers I would need to create separate connections. I am not 100% sure of how to implement it but the general idea is there. For legal and performance reasons all file transfers between clients would be set up using Peer to Peer socket connections. For file transfers between the server and it's clients I have two approaches in my head. One would using only HTTP GET and POST and the other would be the usage of Sockets if a socket connection exists. Which I approach I am going to take, I will have to decide when the basic communication is works.