Disclaimer
The opinions expressed herein are my own personal opinions and do not represent anyone else's view in any way, including those of my employer.
© Copyright 2009
The opinions expressed herein are my own personal opinions and do not represent anyone else's view in any way, including those of my employer.
© Copyright 2009
So within the last few minutes i figured out why i was seeing wrong results when processing the flags for a message. Originally i was getting the flags as part of the message header using an IMAP command like this:
UID FETCH [message uid here] (FLAGS BODY[HEADER])
And it did return the flags, but not accurate ones. In the case of GMail, for every message, regardless of its read/unread status in the UI, the \Seen flag would be returned. This coupled with the fact that the default value for the Seen flag in the Message was true, resulted in every message appearing as though it is new. Not cool.
The solution was to request the message flags separate from the header like this:
UID FETCH [message uid here] FLAGS
Sending this command returns the correct flags, or in the case of a new message, no flags. The only drawback to his approach is the fact that another roundtrip to the server must be performed for each message, minor additional overhead, but additional overhead nonetheless.
Now that the new message detection code is working i can implement a nice quick method of checking for new messages with appropriate callbacks and whatnot. Follow me on twitter @Atmospherian to get updates when i check-in the code.
I know it’s been a while since i’ve posted about InterIMAP and i want to let everyone know that i am planning to continue work on it, fixing bugs and adding features.
Based on the issues and discussions that have been posted on the codeplex site, it seems that some people are still using the old code base in the Synchronous folder tree. This branch will be marked as deprecated going forward as i will not be maintaining it anymore. The asynchronous branch contains far better code, and even supports performing operations in a synchronous manner (blocking the calling thread until the operation completes) so there is no reason to use the old code.
Through some basic testing i did on the code to see what state it is in, i noticed that the new message flag detection code was not working properly, marking messages as new that weren’t necessarily so. This will be the first area I will work on correcting as i recall the message flags that the server returns can vary from server to server (as most things with IMAP tend to be). Once i can correctly determine if a message is new or not i will work on implementing a quick and easy way of checking for new messages either in all folders, or in a specific folder only.
I encourage everyone who uses the library to submit any requests or bugs they find either on this blog or on the codeplex site so that they can be incorporated into the code.
So i’ve beena bit quiet about InterIMAP lately, but that doesn’t mean i haven’t been working on it. Today i will be checking in the latest update of the code which includes some really cool functionality, as well as a new IMAPShell project.
Code Updates
IMAPShell Project
There is a new project in the solution called IMAPShell, and as the name implies, it is a console command shell that allows CLI style interaction with the IMAP library. The goal of the project is to provide a quick and easy way to access message information without having to configure an email client.
It is in its early stages but you can already connect to a server, navigate the directory structure of a mailbox using dir/cd commands and list the messages in a folder using the list command. Command help is fully integrated.
What’s Next
Here is a list of what i will be working on for the next update:
This post will serve as the official requirements documentation for the InterIMAP library. Please comment on this post with your suggestions for further requirements.
Core Requirements
Storage Requirements
API Requirements
So i’ve been reading Steve McConnell’s Code Complete lately, and it’s got me thinking quite a bit about the new work i’ve been doing on the InterIMAP library. When i started working on the new implementation i had this grand view of a well thought-out, well designed system, and to a certain extent, i feel i accomplished that, but in some areas, i fell short.
I initially felt that the new system would only ever need to connect to one server at a time, and as a result i wrote all the manager classes as singletons. I have since come to the conclusion that it may be necessary for multiple servers/accounts to be accessed concurrently, so that will be one of the major requirements going forward. Additionally, it may also be important to some people to be able to store the information retrieved from a mailbox in a full fledged RDBMS, as well as a “local cache” type system, so i will also be working that into the new architecture.
My next post will be a summary of what the requirements of the library will be. Please leave comments as to what you would like to have available in the library.
The time has come for me to continue work on InterIMAP. I have many changes and improvements planned for this library that will improve its usability. In an effort to flesh out my ideas and implementation, i will be putting together different posts for each topic that needs exploration. I welcome all comments and suggestions from those few of you who find their way here.
To kick things off, here is a partial list of the features i am planning to implement or improve (in order of priority):
My next post will go into detail about what i plan to do about the code structure and how i intend to refactor, and reorganize everything.
I am very excited to let you guys know that the re-engineering of the message processing architecture is currently stable and working. My new system for processing message content has not only reduced the number of send/receives from the server to a maximum of 3, but has reduced the code of IMAP.cs by roughly 50-60% making it much cleaner and easier to read and understand.
The original way of processing the message content, which was written by the original author of the code, Rohit Joshi, worked well, except for the fact that it choked on many messages from Gmail, which as i understand is one of the main services that this library is being used with (Exchange is the other). Modifying his code to work with gmail proved nearly impossible due to the extreme variations that can occur in body structure. The process involved first getting the BODYSTRUCTURE of the message, parsing that and then getting each individual BODY[x(.x.x)] section. While this is a valid solution, it is very difficult to write something that can parse the BODYSTRUCTURE in its seemingly infinite variations.
The solution is to pull the full content of the message directly using the BODY[] parameter. The result of this command is every section of the message being return in sequence, usually demarcated with a ‘boundary’ value (except in the case of messages with a single text/plain content type). After this boundary, is the header meta data for that section, which can include the content-type, transfer encoding, disposition, etc. making it easy to parse this header data and properly construct IMAPMessageContent objects.
What does this mean for you? it means that there is now a complete, accurate, and efficient process in place for retrieving the message content from both gmail and exchange.
So whats next? Well, during testing, it became clear to me that for large accounts, having all of the message attachments stored in memory could require a significant amount system resources, much more than an email system should need. So with that in mind i am starting to think about ways to store attachments outside of the main cache file, so that when the cache is loaded, the attachments arent loaded until they are requested.
Major updates in the works this week. After some intial testing with GMail, several show-stopping bugs were discovered in the body structure processing code. Modfying the existing code wasn’t possible as the differences were too severe. Instead i opted to re-write the entire message parsing system to be more flexible and to directly interact with the object model, as opposed to generating XML data first and then using a DataSet to read in that XML which was a cheap hack i employed to get to the message data.
I am not quite ready to upload the newest code as i want to perform some more testing with gmail and exchange before i feel it is ready for a wider audience. There is one bug in particular that is show-stopping with gmail that i am trying to address. Once i get that cleared up we should be in good shape.
Stay tuned for more updates as testing on the re-write continues…
New updates coming this week:
Simple and Advanced Searching
I have added a new search system to the library that should make searching within a folder very simple. There are two new objects in the namespace, IMAPSearchQuery and IMAPSearchResult. The SearchQuery object contains many properties that represent the various search options that can be specified.
The actual Search method is defined in IMAPFolder and takes a IMAPSearchQuery instance as a parameter. The search method returns a IMAPSearchResult object that contains a reference to the query that was used, the folder that was searched and a list of IMAPMessage objects that are the results.
To simplify the search process, several static methods have been defined in the IMAPSearchQuery class that take a single argument, and return an IMAPSearchQuery object, which can be plugged directly into the search method like this:
IMAPFolder f = client.Folders["INBOX"];
IMAPSearchResult sResult = f.Search(IMAPSearchQuery.QuickSearchSubject(“IMAP”));
What the above code basically does is first gets the folder named INBOX, and then calls the Search method on that folder using the one of the quick search static functions to automatically create an IMAPSearchQuery instance to use for the search. The results of the search are stored in sResult and the messages that match the search query can be found in sResult.Messages.
For more advanced searching you can create a IMAPSearchQuery object manually, and specify the terms that you wish to search for. You can specify values for any of the properties that are defined in that class and each will be taken into account during the search.
Currently this system will work only for including all the messages that match the query. There is no support for ‘OR’ or ‘NOT’ style parameters as of yet. The plan i have for this is to add special List<IMAPSearchQuery> properties to the query object, one for ‘NOT’ and one for ‘OR’ where new SearchQueries can be added and processed and included into the main query. I have to experiment with this option and see if it will work well and be intuitive and easy to use.
Local Data Cache
I’ve spent a great deal of time this week working on the local cache system. I’m happy to say that it’s nearly complete. Here is how it currently works:
First, to enable the local cache the developer needs to specify a cache file and cache format in the configuration. This can either be done at runtime, or using the configuration generators.
In the constructor for the IMAPClient class, the configuration is checked to see if a file is defined. if it is, a flag is set, and if autologon is also specified, the system logs in.
Once logged in, the system checks if the specified cache file already exists. If it does, it loads the cache into the object model, and then (if not in offline mode) the system syncs the cache with server (a config option has been added to set the system to auto sync or not). Updates are one-way with the server being the authority. (If the system is running in offline mode, any functions that would change the folder contents or structure are disabled). If the file does not exist, the system then begins the process of building a new cache file. Please be warned that if you are connecting to a large account with many folders and messages this process can definitely take some time (several hours in the worst cases, depending on the location of the server relative to your location, ie. if the server is on a local LAN it will be faster than pulling from the internet).
Once the cache is built, all the data will already be in the object model so no prep work is required to start accessing the messages.
When working in online mode, all of the message and folder copy/move/delete methods will also update the cache when their process completes successfully. This update should not force any special server commands to be run, it should simply serialize the current state of the object model.
Note: this post will be updated as the current state of the feature set changes.
Current Features
Planned Features
Categories
Tag Cloud
Blog RSS
Comments RSS
Void « Default
Life 
Earth 
Wind 
Water 
Fire Light 
Previous Entries
Last 50 Posts
Back