| |
FCIS architecture overview
Introduction
The goal of this document is to provide you with an overview of how FCIS works, from an architectural point of view. By FCIS I'm referring to the entire product, server, client, protocol modules: everything we ship, and everything we'll be shipping soon. You will see what the components of FCIS are, what they do, and how they interact with one another. The focus will be on understanding how things work and why they work that way, rather than on details of configuration and setup. It is hoped that this material will improve your understanding of the product, which should make tasks like configuration and debugging problems that much easier. In this document I will cover:
- What is FCIS?
- How the software components fit together
- Major server subsystems
What is FCIS?
Although the acronym FCIS stands for FirstClass Intranet Server, when I refer to FCIS I'm actually talking about the entire product, including the client. Since the beginning, FirstClass has been a client/server messaging and conferencing system with a built-in directory and advanced security/permission capabilities. As the product evolved, support for multi-server networks was introduced, including message/directory replication and support for third party gateways. The product was further evolved to support custom applications (through the DB API), and to work across platforms. With the release of FCIS, we added a suite of Internet protocols to the system. Rather than adding these protocols directly into the server, FCIS marked the introduction of a new type of server add-on, the protocol module. Internet Services is the protocol module
which provides Internet protocol access to data stored in the FirstClass server. The 5.5 release marked a further evolution of the product, with improvements to the protocol module API that lay the groundwork for the "universal inbox", a concept that is fully exploited by FirstClass Voice Services.
FCIS components
Since we're discussing the entire FCIS product, let's start by looking at the components that make up an FCIS system:
- FirstClass Internet Server (Mac and WinNT)
- FirstClass (the client) (Mac, Win16, Win32, DOS, and UNIX)
- FirstClass Personal (Mac and Win32)
- Gateways (server to server, third party)
- Database extensions (DB API v1.x, DB API v2.x, and RAD)
- Protocol modules (Internet Services and Voice Services)
- Palm conduits
- Utility programs (Designer, POMigrator, FCTools, Installers, Notifier)
How the components fit together
As mentioned earlier, FCIS is a client/server messaging system. The components of the system which have a client/server relationship communicate using the FirstClass Protocol (FCP). Because FCP was designed from the start as a client/server messaging protocol, it can deliver rich messaging functionality across low bandwidth links. Although most components of FCIS communicate using FCP, there are exceptions. For instance, Database Extensions communicate with the server using the Database API, while most FCIS utility programs communicate by updating configuration files which the other programs use.
The basic architecture of FCIS starts with a server, which manages all content and access to that content. Wrapped around the server are various protocols, both low and high level, that are used to deliver the data managed by the server. Sitting outside the server are clients, which use various protocols to access data stored on the server. Let's have a closer look at this architecture, starting at the server and building outward:
The server
The server is the manager and protector of all FirstClass content. It is responsible for accepting connection requests from clients, authenticating the user, limiting their access to the system, maintaining the directory, routing mail, storing all message content, and storing appearance information for conferences.
The server does: Single-store messages and attachments.
Maintain a directory of addressable objects.
Deliver messages to their destination based on addresses.
Handle connections and authentication of user log-ins.
Schedule gateways to run.
Devote processing time to DB transactions.
Limit a users "view" of the system.
Synchronize with other systems (directory and messages).
Support many low-level access protocols (IPX, DDP, TCP/IP, CTB, TAPI, direct modem).
The server does not: Directly support any high-level access protocols, except FCP (the CLUI is an exception to this, but does communicate internally via FCP, so close enough).
Do any rendering of forms or content.
The client
The client provides the interface to FirstClass. It is responsible for finding the server, making the connection, and presenting the data to the user. The important thing to remember is that the client does not store messages nor control access to conferences or features. It is the equivalent of a very clever terminal that has a lot of messaging and conferencing specific features.
The client does: Store and display graphics.
Store some preference information.
Render FirstClass content, including forms, onto the user's screen.
Store information required to find and connect to a server.
Intelligent rendering (e.g. sort/group)
Edit/Update FirstClass content
The client does not: Store messages or conference data.
Limit access to the system in any way.
Gateways
There are 2 major types of gateways, server-to-server and 3rd party. The purpose of either gateway is to transfer mail, conference content, and directory information to another messaging server. Server-to-server gateways connect 2 FCIS systems directly, while 3rd party gateway allow FCIS servers to exchange mail and directories with foreign mail systems. Gateways communicate with the server using FCP, and can therefore run on a different processor from the server.
Database Extensions
The term Database Extensions is actually a bit of a misnomer, they are really more like server extensions. The DB extension was conceived as a way of allowing FirstClass servers to access databases and present their content to the client as if it were content in our own post office. With the advent of the DB 2.0 API in FC 3.5, DB extensions became even more powerful, since they were now allowed to update databases as well. As the API's functionality has increased, we've begun to realize that databases are just one application of this technology. Almost any sort of application can be presented through the DB API, with the RAD product showing just how far this concept can be taken. DB extensions communicate through the DBAPI and run as part of the server process, so they can affect server performance, reliability, and stability.
Protocol Modules (Internet Services, Voice Services, Enterprise Services)
The idea of a Protocol Module is simple: translate foreign messaging protocols into FCP/FCIS on the fly. This sounds a lot like a gateway, but there are some distinctions that make it better. First off, a protocol module must operate like a client, depending on the FCIS it is connected to for all of its message storage and permissions. Tough tasks like translating message formats and speaking 2 protocols are still there, but the need to manage a message store is eliminated and put back into the server where it belongs. This is significant since every protocol module will begin life with 1000 message per second throughput potential, and strong ability to store messages in transit without dropping them. They also get to inherit the gateway's distributed nature, allowing them to be pointed at a server on a separate machine. As well, since
they communicate with the server as a client, it is difficult for them to crash or destabilize the server, and impossible for them to dodge the permissions system.
FCIS has an enhanced connection type for Protocol Modules, sort of a super-client. In addition to usual client things like reading and writing message parts, these enhancements allow protocol modules to do things like direct the queueing of messages in the MTA. Protocol modules also get to store their native addresses in the FCIS directory for later retrieval and validation.
One example of a protocol module is Internet Services. It provides FCIS with standards-based messaging protocols including SMTP/NNTP, mailbox protocols via POP3 and IMAP4, directory protocols via Finger and LDAP, as well as the ability to render FCIS content to HTML/HTTP clients like Netscape Navigator. Another example of a protocol module is Voice Services. It provides services similar to legacy voice mail systems, but with additional capabilities, such as listening to your voice mail from within FCIC or from a web browser.
From the discussion above, you can see that Protocol Modules actually fall into 3 sub-categories:
Gateway Services: (e.g. SMTP/NNTP/POP "sucker") These move bulk content in/out of FirstClass.
Client Services: (e.g. HTTP/POP3/IMAP4/FTP) These render post office content to alternate clients.
Directory Services: (e.g. Finger/LDAP) These render directory content to alternate clients.
Palm Conduits
Another type of module that can attach to the server is represented by the Palm Conduit. The Palm Conduit is like a protocol module, but instead of running a single instance back near the server, it runs on a client machine. The Palm Conduit runs when a Palm Pilot is put in it's cradle to synchronize. It connects back to the users account on their FC server using FCP, and synchronizes a particular Palm database with the corresponding FC object. Since the user information is configured on the Pilot, any cradle with our conduit installed can be used by any Palm to synchronize that user's data.
The Big Picture
This diagram shows everything we've looked at so far on an active FCIS system. The server can have it's functionality expanded via a RAD application using the DB API. The Enterprise Services protocol module is present, sharing messages with a Microsoft system. A Palm conduit is connected, synchronizing a users Palm databases with the objects stored on their desktop on the FCIS system. Another downstream FCIS server is connected using the gateway protocol, synchronizing conference and directory content, implementing a distributed, multi-server system. Internet Services is in place, providing access to FirstClass content through a variety of Internet protocols. Voice services is there, accepting call answering sessions from the PBX, and allowing users to login via the telephone to check for voice, FAX, or text mail. The client is talking
directly to the server, provided the richest messaging experience.
 Utility programs
Other Stuff - There is plenty of Other Stuff® that is part of the product. Some of the significant ones are:
Installers - Gets all the stuff onto your machine
Designer - Allows FCIS to be customized (forms, icons, graphics)
FC Personal - Offline capabilities
FCTools - Server licensing and configuration
Notifier - Incoming mail notification
POMigrator - Moves PO's from Mac to NT
Major Server subsystems
Since the server sits at the center of our discussion, we'll have a quick look at what subsystems it has and how they compare to competing products. Note that the server does have a lot of other subsystems, but these 5 represent the major systems with direct analogs in almost all other mail systems. Another important point here is that there are core subsystems and then applications built on those systems. Using this technology, it is possible to add functionality to the server while still building atop a fast and powerful foundation.
Collaborative Store
The first thing that comes to mind when collaborative servers are discussed is the collaborative store. A collaborative store is really a secure and compact representation of messages on the system, sort of a database of discussions. There are various approaches to collaborative store technology taken by different vendors, but they usually fall into one of two camps: database, where all messages are stored as objects in an indexed database, or file-based, where messages are spread into sub-directories, each of which represents a user or destination. Although you might feel FCIS is the latter, the truth is that it is a hybrid of the 2 approaches. It stores a "message oriented database" for each mailbox. This "database" is optimized for two major operations: (1) Building a complex summary of the messages in a
mailbox. (2) Adding new records to the database (message delivery).
FCIS's hybrid approach puts us in a best of both worlds situation when it comes to server speed and features. The single copy collaborative store both reduces disk space requirements and increases delivery speed as deliveries are done by creating a link to the message. Distributing the post office files throughout a directory tree gives us strong repair/backup facilities, low file damage exposure, and reduces I/O bottlenecks, all of which are potential problems with pure database solutions. The "message oriented database" also gives us many of the advantages of database-style post offices, such as high speed access to advanced features and built in permissions systems oriented toward messaging tasks. Many pure file based systems must scan individual message files to implement these types of features.
The Message Transfer Agent
The MTA is another universal concept in messaging servers. In general an MTA routes and delivers messages based on the addressing scheme the system uses. Ours is no different in this respect, and does exactly this. Messages are delivered to users who reside on the MTA's server, and routed to users who reside on other servers. The first step in this process is determining where to send the message. In the case of the FCIS server, the address is looked up in the directory. If it is a local user (similar for conference, routes, etc.) a FID (file link) to that users mailbox is located in the record returned from the lookup. Open the "messaging database" (Eninfo) file at that FID and add a pointer (FID) back to the message being delivered. No messages copied, just a fast lookup (see directory below), a small record append, and you're
done. This is why we are orders of magnitude faster than some other messaging servers.
The Directory
At the heart of the messaging server is the directory. Through this, addresses are resolved, and users are authenticated. Some messaging servers take the piggyback approach to this, using their native OS's directory to manage log-ins and addressing. These solutions are generally unportable, and are often slow since the OS's directory wasn't designed with email in mind.
The FCIS directory is an "address oriented database". It is optimized to perform extremely fast look-ups (1 disk op) and also to produce fast sorted lists based on the search criteria (multi-match lists) with very low disk I/O. (e.g. 1 disk op to find and display 100 names beginning with "s" - contrasted with a typically Relational Database cost of 1-3 disk ops to resolve name, plus a disk op per name to get the details - a 100:1 ratio)
FCIS is based on a high performance directory system, that stores each object multiple times for speed of access and redundancy. The FCIS directory is based on 3 indexes considered key to message delivery and access control: the User ID index, the Client ID index, and the Name index.
When a new user is created, some critical information is entered or created to represent that person. The client ID is a monotonically incrementing number assigned by the server. This number (4 bytes) is the most compact way to represent a user and so is used when the server needs to store a reference to someone.
The user ID is generally chosen by the admin or the user himself, as a compact, human readable representation of the person. The login process requires a user to provide the user ID, and so this is the index accessed when users log in to FCIS.
The user's name is their actual, human name and can include all sorts of silly characters. It is separated into first and last names and these are stored as keys in the name index. The directory indexes first, last and multi-part last names, so you can enter "doosen" and match "Julie Van Doosen". It is also aware of the structure of names. So "J Do" will match "Julie Van Doosen" but not "Doris Jules". "Do" would find both names, however. In general, people can remember your name, or at least some of it, so FCIS addressing is usually done by entering some or all of a person's name. The name index is used whenever an address needs resolving.
Security Systems
The FCIS server has a multi-layered approach to security and permissions. First off, we assume that the server machine is physically secure and protected from network attacks. This is generally an easy/safe assumption for a server machine, since without this, no server can really be considered secure.
Once the machine itself is OK, our server takes over and things become secure in a hurry. Log-ins are via FCP, not piggy-backed on any native OS login. This means a user must get authenticated through our directory and through their user profile before they can get in. Once in, one of the key deterrents to snooping is data hiding. The users foothold is their desktop, so anything they can't see through that, they cannot get to. There is no way to jump to a conference directly.
After this, the permissions system comes into play. When they login, users are looked up in the directory, and their Privilege Group membership is examined. This forms a privilege list for the user which then limits the types of features and operations that user can access. Another type of security kicks in when user try to access conferences or gateways. The permissions settings on the object limit what specific users or groups can do in that area. Finally, conferences and users can be organized into groups, allowing them to inherit a generic permissions setup.
Protocol
At the core of most messaging systems is a "protocol" for connecting the clients to the servers. We use FCP to perform this function, which is a rich client/server protocol for remotely manipulating objects in the collaborative store. Some competing products use "protocols" like: 1) leave the file under a rock 2) make remote procedure calls 3) use a suite of simple text-based protocols that force an architecture of download and scan to implement features. Any of these solutions can only be called client/server in the most generous interpretation of the term. A proper client/server protocol (like FCP) should promote efficient access to stored object, with all object updates occurring on the server machine. Any local storage of collaborative objects breaks universal access and is not truly client/server.
If you would like to see additional features by the author of this article, click here.
|