Inside SVS :: How (and Why) fslx.sys Tracks Windows Processes

The core technology in SVS is known inside Symantec as "Fortress". The Fortress file system filter driver -- fslx.sys -- is the traffic cop that controls interaction between the operating system and applications in SVS virtual layers.

In the interest of simplicity, Fortress is architected to leverage and extend native Windows functionality wherever possible. This approach ensures that virtualizing an application has the minimal possible impact on the appearance, behavior and supportability of that app. For example, fslx.sys maintains the barrier between the real and the virtual by tracking Windows process trees -- keeping things in order the same way that windows itself does.

What's a PID?

Wikipedia defines a "process" as "an instance of a computer program that is being sequentially executed." All running processes on Windows get a PID (Process ID) #. You can see the PIDs for the major user space processes when you bring up Task Manager > Processes, select View > Select Columns... and check the PID box. Everything done by a process (including all file system and registry calls) is stamped with the process' PID. When a process kicks off another process, the new process gets a PID as well, and is tracked by Windows as a child process. A parent and all of its children, grandchildren, etc., form a "process tree". The process that starts the process tree is called the root process.

You can best see how this looks on any Windows machine by running Process Explorer, a totally awesome tool (essential for SVS development and support) that MS acquired when it bought Sysinternals. It can be downloaded from the MS Web site or from Download.com.

How SVS Uses This Information

Fslx.sys watches all process trees on the system and maintains a map in memory. When a file system operation is called, for example, SVS looks at what process is making the call and knows what process tree it is part of. So it knows what the root process is. (Very often, the process has no parent or children, in which case it is its own root.) SVS makes decisions about when/what to redirect (and to/from where) based where the the root process was launched from. That is, was it launched from a file that exists in a layer -- and which layer -- vs. a file in the base.

Example: When a File Create function call goes to the Windows IO Manager, SVS checks the PID of the process making that function call. It then checks the table to see what the process tree root is for that PID. Let's say the root process was launched from an executable in the read-only sublayer of application layer XYZ. SVS then knows where the new file needs to be created -- within the writable sublayer of application layer XYZ. There's some more complexity when a process tree "crosses over" from the base to a layer, or from one layer to another, but this is the basic concept.

How Process Tracking Is Used During Capture

When a capture is being done and a new read-only sublayer is being built out... Well, it's actually one of the most basic things that SVS does. The root of the process tree is the executable that you selected in the Browse dialog for the Single Program Capture. All writes made by that process and its children go into the read-only sublayer of the new layer. In contrast, a Global Capture doesn't worry about tracking processes -- it redirects all writes into the new layer's read-only sublayer.

The reason why every once in a while you have to use Global Capture to get the desired result is simple. It's possible to "jump the process tree" (my own term). That is, for a series of process launches to not be tracked by Windows as one contiguous process tree. The most common example -- and really the only one that I'm aware of that is a real-world concern -- is when an install process calls a Windows Service (other than the Windows Installer Service) and has the service do something on its behalf. Services are not tracked by Windows as children in a process tree.

Examples: Let's say an install routine is creating a printer (Adobe Acrobat being both the first example we encountered and the most common one that comes up). When creating a printer, you have to ask the Print Spooler Service to do some work. A Single Program Capture will see that as background noise, ignore it, and the printer does not get created correctly in the new layer. That's why Global Capture is recommended for Adobe Acrobat. VMware Workstation is another example -- you can put it in a layer, no problem, but you have to do a Global Capture because the install calls the Network Connections Service to create the VMware network interfaces.

Conclusion

The great part for SVS development is that we didn't have to write any of this stuff. All the needed info was just there in Windows, waiting for SVS to come along and use it to make magic. This is one example of what I mean when I say that SVS extends the plumbing that is inherent in native Windows. MS put the process tracking system in place for a lot of reasons, but it's core to SVS being able to do what it does. It makes redirection and prioritization -- and the resultant normal visibility of virtualized apps -- possible.