Re: Linux filesystems -- Journaling, ACs, Features, Win v. UNIX

["Bryan J. Smith" <b.j.smith@ieee.org>]

I'm going to top-post.

To start, I recommend _against_ getting into such "meta-discussions."
The people starting them are often on a political agenda, and they are
many times very _ignorant_ of what the capabilities of _standardized_
UNIX-POSIX in general are (let alone Linux).

But I'll heed the discussion, and start with a break-down the "criteria"
of possible comparison:  

 0.  History (including 32-bit v. 64-bit)
 1.  Journaling and Consistency Checking
 2.  Access Control
 3.  Features (Compression and Encryption)
 4.  Application Considerations


===========
0.  History
===========

- NTFS

NTFS is based on IBM's HPFS design from OS/2, but is very different.

NTFS is largely a complete rewrite by the former Digital VMS engineers,
with some major changes from HPFS for NTFS.  Access controls,
compression and, in later NT versions (5.x aka 2000/XP/2003),
encryption.  Despite advantages, in the end, there is heavy FAT
influence in design, a "less-than-reliable" journaling logic (that has
bit me in the ass *HARD* on more than one occasion) as well as the
chronic plague of Windows heirarchial layout as we will see.

Another major note on NTFS is how it stores its security references. 
Because of this, NTFS filesystem can*NOT* be "moved" between systems --
not even other NT/2000/XP/2003 systems _unless_ either "centralized"
(e.g., domain) SIDs are used _or_, to a limited extent, "dynamic disks"
are used.  This is why not only Linux cannot "write" to NTFS filesystems
reliably, but even _other_ NT/2000/XP/2003 systems as well -- the issue
is with NTFS _itself_.

[ VMS-OS/2 SIDE NOTES:  
The original NT design team was a majority of formal Digital VMS
developers.  Microsoft was able to prevent a lawsuit by Digital by
promising Digital premier access to NT as well as a native Alpha port
(although that is another, very sad story where Digital only ended up
designing the 64-bit version of Windows 2000/XP being used today for
MIcrosoft).
Although OS/2 was also from Microsoft, Microsoft _only_ wrote the
"Program Manager" interface which was chucked in 1992 in favor of the
IBM designed "Workplace Shell."  Even though, Microsoft retained
ownership of all OS/2 source code through 1993, and then wessel'ed the
rest of the Warp code out of IBM with the Windows 95 licensing agreement
some 2 years later, in reality, "Chicago" or MS-DOS 7 / Windows 4 which
became "Windows 95," actually has far more OS/2 code in it than Windows
NT ever has. ]

Linux offers 4 journaling filesystems today, Ext3, JFS, ReiserFS and
XFS.  Different filesystems have varying levels of IEEE POSIX
standardization.

- Ext3

Tweedie's Ext3 is based on Ext2, with its nearly 10-year static design,
proven fsck and "conservative" journal logic, is the favorite among many
major distributions.  Newer Ext3 releases support some of the latest
IEEE POSIX standards, including POSIX Extended Attributes (EA) and
Access Control Lists (ACL).

- JFS

IBM's JFS, is ported from OS/2 non-UNIX _instead_of_ AIX UNIX because of
UNIX licensing issues that IBM was clearly concerned with almost 4 years
before the SCO lawsuit (ironic, eh?).  That meant it had to be
"retrofitted" with IEEE POSIX features not in the OS/2 flavor, causing
JFS on Linux to lack a _number_ of interfaces from the get-go (e.g., NFS
compatibility issues).  It also, more of my opinion (i.e., yours may
favor) seems to inherents a lot of "bad approaches" that are clearly
HPFS-influenced.  But it was a true 64-bit filesystem from the get-go,
including on Linux.

- ReiserFS

Hans Reiser/Namesys' ReiserFS is a quantum shift in filesystem design. 
Ironically, despite the rapid development pace, including _massive_
internal filesystem structural changes on a regular basis, manages to be
a very reliable filesystem from an "on-line" standpoint.  More personal
experience, the journaling logic and recovery is most excellent,
including knowing when the journal should not be trusted (a very
important consideration).  The downside is that the Namesys team (let
alone even popular distro developers) doesn't seem to keep the
"off-line" tools quite "in-sync," leading to some of the worst
filesystem disasters I've ever seen.  And because of the focus of
ReiserFS, IEEE POSIX standards integration is not at the top of their
priorities.

- XFS

SGI's XFS is like IBM's JFS port in that it is a true 64-bit
filesystem.  But unlike IBM's JFS, XFS is a direct port from a full UNIX
to Linux -- namely from SGI Irix to Linux.  Better yet are the facts
that the structure of XFS on Linux is the _exact_same_ as its 10-year
old implementation on Irix and, other than a brief errata in the 1.0
(fixed in 1.1) release, has a near flawless record.  More on the feature
side, XFS on Linux had POSIX EA/ACL support from day 1, as well as
_full_ quota support (not even Ext3 did) and was 100% Linux kernel
interface compatible (e.g,. NFS worked from day 1).  The XFS team also
laid down much of the "groundwork" for POSIX EA/ACL support in the
2.5/2.6 kernel (Linus agreed on the IEEE POSIX EA/ACL API as of 2.5.3),
of which has been backported to 2.4 for use by both XFS as well as Ext3.

XFS has not been included in the stock 2.4 kernel not for reasons of
maturity.  It has to do with the _massive_ number of kernel subsystems
that simply do not exist in kernel 2.4 that are required for XFS. 
That's why the XFS module is "huge," because it is loading a large
portion of the supplemental kernel that is required.  Kernel 2.6 now
features these capabilities, and all Linux filesystems benefit.  As
such, XFS is a two-part development, one from a kernel subsystem
standpoint (for 2.4 only), and another for the actual filesystem driver
(for both 2.4 and 2.6).

- POSIX:  32-bit v. 64-bit

There is a _lot_ of confusion on whether Linux filesystems are "32-bit"
or "64-bit," and many Windows admins naive to UNIX/Linux like to say it
is the former, because NTFS is the latter.  The problem isn't Linux or
UNIX per-se, but POSIX itself, and limited to _select_ filesystems for
Linux.

POSIX is 32-bit or 64-bit based on the architecture it is running on. 
This means on IA-32 (aka x86), it is 32-bit.  On IA-64, x86-64 (aka
AMD64), Alpha, MIPS4000+, etc... it is 64-bit.  Issues with the former
are where the infamous "2-4 GiB" limitations come from.

Turn of the millenium, the IEEE POSIX 1999 introduced the concept of
"Large File Support" (LFS -- not to be confused with other acronyms of
the same, like "Linux from Scratch").  LFS allows 32-bit POSIX systems
to use 64-bit filesystems.  Thus, LFS is how previously 32-bit
filesystems on 32-bit architecures could now use 64-bit filesystems,
like they were 64-bit architectures.

You _must_ run a Linux kernel, standard libraries (e.g., GLibC) and any
other support libraries all with LFS-compliant code for a "64-bit clean"
program to support 64-bit filesystems.  In Linux terms, this has been
kernel 2.4 and GLibC 2.2 -- circa Red Hat Linux (RHL) 7, although
patches _were_ available for kernel 2.2 and GLibC 2.1.  In addition, any
network services _must_ also support 64-bit transfers -- e.g., NFS v3
and, (for bug-free operation), Samba 2.2.7a+.

You'll be happy to note that both IBM JFS and SGI XFS have _always_ been
64-bit, both on their native platforms (OS/2 and Irix, respectively), as
well as Linux.  In fact, that's why JFS and XFS were not really "usable"
until the distros came out with kernel 2.4 and GLibC 2.2, even though
the were being worked on prior, including even some modified kernel 2.2
releases with GLibC 2.1.

Ext3 as of _stock_ kernel 2.4.15 is 64-bit/LFS compatible, including
full NFS v3 support.  Support for Ext3 was available from vendors prior
as well (I even had a kernel 2.2.19, GLibC 2.1.3 setup from VA Linux
that supported 64-bit/LFS, including over NFS v3).  ReiserFS v4 is fully
64-bit, although ReiserFS v3 in kernel 2.4.1+ can support some
64-bit/LFS structures.


=======================================
1.  Journaling and Consistency Checking
=======================================

One thing to remember about journaling and "on-line" filesystem
integrity check (fsck) recovery is that it does _not_ improve recovery,
it only _expedites_ it.  Some filesystems, like MS FAT, care less about
ensuring filesystem integrity, and there is nothing worse than writing
to an inconsistent filesystem, as more damage to the filesystem may be
done.  Only a full, "off-line" fsck can _guarantee_ the filesystem is
consistent, but takes a long, long time.  Journaling is the concept of
trying to maintain some atomicity in writes, to either avoid completely
any fsck, use a "on-line" fsck or reduce the time it takes to do an
"off-line" fsck.

NTFS, like HPFS, has a _quality_ "off-line" fsck in CHKDSK.  CHKDSK is
one of the slowest, but thorough "off-line" fsck in the business.  But
CHKDSK is not where the issue is with NTFS.  The problem with NTFS is
that is _always_ goes to a "journal replay."  That means that NTFS, like
HPFS, always _assumes_ the journal is _always_ valid.  The only way to
avoid this is to check a box in NT/2000/XP/2003 to do a full disk check
at the next reboot.  But if you're NT sysetm has crashed, there is
little means to ensure this (other than maybe using a Linux boot disk to
modify the NTFS structures to "flip the bit" to force this "off-line").

Twice have I had mission critical NT servers _bomb_ because it did a
journal replay and the filesystem was _toasted_.  IMHO, the NTFS journal
logic really needs something that checks its journal _first_ and _then_
offers to do a full CHKDSK because "hey, things don't look right here." 
It _will_ sometimes do that for you _after_ it has replayed the journal,
because it is still not consistent, but if the journal is really
"screwed up," that "journal replay" may toast the disk.

The "next generation" Microsoft filesystem for NT 6.0 (Longhorn) looks
to be little more than NTFS with some features "slapped on" in its
current 2004 "preview" incarnation.  There is little reason to believe
this will change, as Micrsoft is unlikely to attempt to rewrite the
proven CHKDSK off-line recovery tools for the current NTFS structure
(which is very much like HPFS from a "raw" structural standpoint).

- Ext3 

Ext3 is fairly unique versus all the filesystems here.  In addition to
offering the common "meta-data journaling" mode(s) of others, Ext3 also
offers a "full data journaling" mode.  The difference is that Ext3 is
offering atomic _data_ writes (i.e. the actual data being written), in
addition to atomic "meta-data" (i.e. allocation/block info, file
indicies, directory info, etc...).  Normally you only need the latter to
keep a filesystem consistent without a full fsck.  But with full data
journaling like Ext3 offers, if you use a NVRAM journaling solution
(e.g., add-in PCI card), you can almost prevent _any_ data loss period
(as well as speed up network filesystems that guarantee "sync" before
returning control -- e.g., NFS v3 in "sync" mode).

Ext3 itself is just Ext2 with journaling and an off-line journal replay
(no on-line recovery option), keeping the same, on-filesystem structure
since the mid-'90s.  In fact, when Ext3 needs to "drop down" to a full
"off-line" fsck, it simply runs the fsck for Ext2 (right from the same
e2fsprogs package -- although be sure to have a current version).  And
not only are its off-line tools trusted, but Ext3's on-line journaling
logic is _very_conservative_.  If it doesn't like its journal, it will
simply fail to replay it, prompting for a full fsck.

Yes, it's the slowest to journal replay, offers no on-line recovery and
no extra "features" other than POSIX EA/ACLs, but it is trusted and
reliable.  Although Ext3 offers two different "meta-data" modes,
"ordered" (default) and "write back," the default should not be
changed.  Switching to "write back" will give you increased performance,
but it may cause additional data on the disk to be trashed.  This is
because onyl "ordered" guarantees that meta-data related to data being
committed with be touched, and in a way that will not affect other
data.  "Write back" may modify structures that may cause existing data
on-disk (not related to the write) to become "inconsistent."

- JFS

JFS is the filesystem I have the least, personal experience with.  I
quickly dismissed JFS early on because it was ported from OS/2 instead
of AIX, and I even lambasted IBM for doing that, because the OS/2
version lacked a lot of UNIX-design (e.g., no kernel NFS support in the
first few major releases) unlike the version from AIX did.  Little did I
know at the time (1999) that the reasons were licensing related -- a
further indicator to me that SCO has no footing (because IBM _was_
concerned about IP transfer from AIX to Linux, even _years_before_ the
lawsuit).  Yes, OS/2's JFS is based on AIX's JFS implementation, but it
is a complete rewrite.

And this is where the HPFS/NTFS influence comes in.  Although JFS has a
good set of "off-line" recovery tools, like its HPFS predecessor, it
appears to have the same "replay by default" journal logic that I
dislike with HPFS and NTFS.  It then tries an on-line recovery after the
replay.  I have heard of at least 5 people bitten by this on not only
Linux, but AIX systems as well.  The case where the journal replay was
not successful, and the full fsck was unable to make sense of anything.

JFS is where most of my ignorance is at as of late, as I have not kept
up with kernel 2.6 integration.  I have also tried a couple of distros
with JFS (e.g., Mandrake), and been extremely disappointed.  In those
cases, it was not so much JFS itself, but the distro vendor
*COUGH*Mandrake*COUGH* who did not even bother getting the proper
off-line tools to match its kernel implementation.  Doh!

- ReiserFS

ReiserFS is also a filesystem where I have limited, personal experience
with, although I have _extensive_ experience in helping others try to
recover ReiserFS filesystems.  Unlike JFS, ReiserFS has some
_impressive_ on-line recovery capability and is not "shy" to reject its
own journal and force off-line recovery.  And that's where I've run into
issues.  Not only do many distros *COUGH*Mandrake*COUGH* _fail_ to
include the _correct_ off-line recovery tools -- I have in no less than
2 cases went out and retrieved the _proper_ version of the off-line
recovery tools for the _specific_ ReiserFS kernel implementation and
they have _toasted_ the ReiserFS filesystem.

This issue of "off-line recovery" of an unclean ReiserFS filesystem that
canot replay its journal is _unlikely_ to change anytime soon.  Hans'
own attitude of "filesystems should be redesigned every 5 years,"
combined with the ever-changing structure of ReiserFS (even if v3 in the
2.4 kernel does not much), is basically mutually exclusive with the
creation of solid off-line recovery tools.  It takes a good 2-3 years
just for the off-line fsck tools to mature.

Heck, even Microsoft's "next generation" filesystem for NT 6.0
(Longhorn) with integrated database capabilities is actually
_nothing_more_ (at least in the 2004 preview and code leaks) than the
proven NTFS structure with added features "slapped on" and not the
"revolutionary change" that people expect.  The reason is simple,
radical changes in filesystem designs require years for their off-line
recovery tools to mature into proven ones.

- XFS

SGI's XFS was an instant adoption by myself.  With full quota, POSIX
EA/ACL, Samba ACL (thanx to Allison's SGI employment), Linux
kernel-compatible interfaces (e.g., NFS) and other support off-the-shelf
when it was released for Linux in 2001, it was a hit in my company. 
Probably the biggest seller for me was the Ext2/Ext3-like proven
structure, just as old at about 10-years.  In fact, given the added disk
label (i.e. partitioning) support in Linux, you could _seemlessly_ move
disks between Irix and Linux systems without issue.

XFS uses a combination of journal replay and an on-line, transparent
fsck with a manual off-line fsck (XFS_Repair).  Other than one case
where I had a physical disc error, the on-line, transparent fsck has
worked flawlessly.  XFS does a quick test when it mounts to see if its
journal can be replayed and, if not, if an on-line recovery will
succeed.  It then mounts and procedes with either in "real-time" --
i.e., XFS will intercept any writes to any suspect "flagged blocks" and
address them, while its doing a journal replay or on-line recovery.  It
is a truly advanced filesystem in design, licensed under the GPL for a
reason -- because other vendors want it for their own OSes (e.g., the
BSD guys have disliked the fact that its not BSD, but understand why --
it regularly comes up on the XFS lists).

About the only issue XFS had was with its on-line, transparent fsck was
in version 1.0 (fixed in 1.1).  I ran into it myself, and it toasted one
of my /var filesystems (clearly the place where this was most likely
going to occur for most).  It was only the Linux version of XFS, and the
porting error from Irix was discoverd in short order.  I know no one
where it hit a data or other filesystem (/var and /tmp seemed to be the
only).


==================
2.  Access Control
==================

- NTFS

NTFS access control is engrained into NT's concept of a Security
Identifier (SID).  SIDs can be 6 levels thick, although most
applications in the Security Account Manager (SAM), which is the "object
database" regardless if you are using a local system, a legacy CIFS
domain or a new ActiveDirectory domain, only go two (computer/domani +
object) or three (domain + user + other) levels.  Despite the extensive
differentiatation, the only difference beween a "workgroup" and a
"domain" of _any_ Windows version/directory implementation, is whether
SIDs are maintained per system or network-wide.

One confusing aspect of SIDs is how they relate to just any "reference
number" in another filesystems.  Other filesystems simply store these
numbers, be they for users, groups, etc..., and it is up to the mounting
system to understand them.  Those that use NIS, LDAP or other directory
systems that can relate the numbers to an object do so, otherwise the
number is simply reported (e.g., if the user/group was local).  No harm
done.  SIDs are very different, and require a SAM to map these SIDs to
something useable.  In the case of a domain, the SAM is just
"centralized" for all users to use.  But even so, the local C:\WINNT
directory is still filled with SIDs for _local_ SAM references, and is
an issue in itself for even a _domain_controller_.

To simply "spit out" a SID attribute back to a directory or file without
connecting the relationship in a NTFS filesystem can _corrupt_ that
filesystem.  Because all NTFS filesystems have SIDs, if a system that
mounts a NTFS filesystem cannot reference _all_ SIDs, again, it runs the
risk of corrupting the filesystem if it writes to it.  This is
_regardless_ of OS, be it Linux or even the _exact_same_ version of
Windows NT as the creating system.

The "workaround" is to use NT 5.x's (2000/XP/2003) concept of a "dynamic
disk."  The "dynamic disk" is a new type of "disk label," different from
the traditional MS-DOS "disk label" of primary, extended, logic, etc...,
but not unheard of in the non-PC world.  As much as people have
complained about "dynamic disks," they serve one very important
purpose.  They have a "hidden area" where these SID maps can be stored
_outside_ the SAM database of the system.  Therefore, any OS that can
understand dynamic disks and reference this "hidden area" on them, can
safely mount and write to NTFS filesystems.

As far as I know, there is no effort currently to support SIDs in IEEE
POSIX.  But projects like Samba and OpenLDAP do have some facilities for
emulating SID functionality in their native, non-SID stores (e.g., map
SIDs to LDAP-objects) as well as emulate "Access Control Entries" (ACE)
that are expected by Windows clients (e.g., Samba makes POSIX EA/ACLs
look like they are simply NTFS/SIDs over CIFS-SMB).  This of course
requires Samba to be running on a UNIX platform with a filesystem that
not only supports POSIX EA/ACLs, but in a way Samba understands.

- Ext3

Like traditional UNIX filesystems, Ext3 supports legacy IEEE POSIX
(pre-1988) UNIX ACLs with the user, group and other permissions.  But
during kernel 2.4 development, a portion of the Ext3 team worked on
supporting IEEE POSIX 199x Extended Attributes (EA) and Access Control
Lists (ACLs).  Red Hat even shipped a Red Hat Linux (RHL) 8 beta with
them enabled on Ext3, but disabled them for the final release after
numerous issues were experienced.

Linus approved the POSIX EA/ACL implementation for kernel 2.6 as of the
2.5.3 development version some 18 months ago.  It was based largely on
SGI's XFS modifications.  There are both kernel functions and user-space
utilities for "getfacl" (get file ACL) and "setfacl" (set file ACL).

- JFS

IBM's JFS supports EA/ACLs even on OS/2.  I do not believe they are IEEE
POSIX-compliant ones, but I have not kept up with JFS developments.  As
of kernel 2.6, all filesystems should have support for such directly in
the kernel, although actual storage and implementation is up to the
filesystem.  Anyone kept up with JFS in this regard?

- ReiserFS

Like JFS, I have not kept up with POSIX EA/ACL support for ReiserFS. 
ReiserFS certainly has the capability, but it's all about the focus. 
With kernel 2.6, I'll safely assume ReiserFS v4 has been modified to
support the POSIX EA/ACL support.

- XFS

XFS on Linux has supported the full IEEE POSIX 199x EA/ACL standard
since day 1, based on the Irix implementation directly ported over.  The
implementation was a custom, GPL addition to kernel 2.2/2.4.  This like
many other XFS "additions" to the kernel itself, that are _not_
XFS-centric, are what keep XFS from being a standard filesystem in
kernel 2.4.  These subsystems, modified, have been included in kernel
2.6, with much help from SGI (among other projects, like Non-Uniform
Memory Architecture, NUMA, which the AMD x86-64/Opteron platform
directly benefits from).

SGI XFS 1.0 and 1.1 releases were using the legacy SGI add-on
implementation.  SGI released XFS 1.2 shortly after Linus' 2.5.3
decision, which is based on a 2.4 backport from 2.6, making the XFS
filesystem driver in 2.4 fully compatible with 2.6.  Again, SGI
maintains XFS in two portions, one which is used for only 2.4 (missing
kernel 2.4 subsystems) and another which is used for both 2.4 and 2.6
(filesystem driver).

The tools remain the same, "getfacl", "setfcal" (get/set file ACLs),
that have existed forever in Irix and the pre-2.5.3 Linux XFS
implemenation, as well as the new XFS 1.2+ versions and the 2.6
integrated XFS filesystem.  Samba has _fully_supported_ XFS EA/ACLs on
Irix before either Ext3 or XFS was supported on Linux, and the latter
benefits from this.


=========================================
3.  Features (Compression and Encryption)
=========================================

- NTFS

Microsoft offers two distinct features in NTFS.  One was original, and
one was added.  They are compression, using the same, royalty-free LZ77
algorithm as in PKZip and GZip, and encryption, which was not added
until NT 5.x (2000+).

Both compression and encryption are transparent, including via network
filesystem access.  The compression/encryption portion is _always_ done
by the system that is _local_ to the NTFS filesystem -- e.g., the server
itself.  That way it is compatible with legacy MS-DOS 7.x kernel
platforms like Windows 95, 98 and ME, which do not support this
functionality natively.  They also do compression/encryption on a
"per-file" basis, and not compressing the whole filesystem.

Encryption aka "Encrypted Filesystem" (EFS) can be implemented in a
couple of different ways.  In a nutshell, if you are using the default
approach of EFS, and not using any Kerberos or certificate-based
capabilities, then the NT Administrator password is stored on the system
somewhere in a _reversible_key_.  That means by default, the "master"
EFS key _could_ be hacked, even if the system was off-line.

Using either Kerberos or Certificate-based EFS is highly recommended,
both from a security as well as a recoverability (e.g., if the local
Administrator password is lost, you'd have to find the reversible key
just like any hacker) standpoint.  Key escrow is best handled by either
Kerberos or a X.509 Certificate.  Again, using the default EFS approach
*IS* "hackable."  Once that key is found (and it can be), _all_ EFS
files for _all_ users on the system can be decrypted, just from that
_one_ key.

EFS should _not_ be considered a true Public Key Infrastructure (PKI)
implementation for file encryption, and it is proprietary at that.


- Linux (all filesystems)

Linux offers a variety of different capabilities for compression and
encryption.  Linux, like most UNIX platforms, prefer to stick with
per-file compression/encryption.  Transparent access has been added to
some drivers -- e.g., transparent LZ77 (GZip) decompression in a
modified ISO9660 (CD/DVD-ROM) driver for the kernel (it sees any .gz
file as a "regular" file when accessed).  The kernel also integrates
BSD-licensed LZ77 (GZip) and BWT (BZip2) libraries, and its only a
matter of modifying the drivers for the support.

[ SIDE NOTE:  I have personally suggeted kernel developers look at using
LZO (LZOP), a GPL (but not LGPL) algorithm/library, as it is a real-time
compressor that is 3-8x faster than LZ77/GZip, with only a 10-15%
compression ratio difference, or 8-15x faster than BWT/BZip, with only a
15-25% compression ratio difference. ]

More on the encryption end, Linux offers from various symmetric and
asymmetric implementations, such as DES, AES and OpenPGP-compliant
solutions, to X.509 certificate and other standard services.  OpenSSL is
at the heart of most of these solutions.  The small education of users
in using GnuPG (GPG), which is _far_more_secure_ than EFS (especially on
an individual-by-individual basis).  Otherwise, only a few companies
like Entrust offer a _true_, 100% standards-based Public Key
Infrastructure (PKI) solution, one that runs on a variety of platforms,
including Linux.

>From a filesystem standpoint, using transparent encryption/compression
is typically done via the "loopback" device, or a program with similar
functionality.  Filesystems that need to be completely secured should be
mounted as such.  But this level of "transparency" does require a little
effort.

In a nutshell, proponents of MS EFS should realize that "security" and
"ease-of-use" are _mutually_exclusive_.  EFS' default mode should
_never_ be used, and only using Kerberos or per-user Certificates are
ideal -- assuming you don't believe your DC will be hacked.

Which is my final point:
Using a "true" Kerberos solution, where your Kerberos Distribution
Center (KDC -- i.e. key server) is _only_ handling that facility and not
10,000 other things (like a Windows Server DC) is far more _ideal_
IMHO.  Especially if that KDC is behind a network firewall, and the only
thing getting in and out is port 88.


===============================
 4.  Application Considerations
===============================

Ahhh, my favorite name of the game.  Forgetting the OS, what do the
applications of the OS do?

- NTFS

NTFS suffers from *2* _chronic_ problems with Windows.

A.  The _requirement_ of a default application "startup up" directory
B.  No strict separation of OS, temporarily files and data

"A" is the first thing any company runs into when porting a Windows app
to UNIX via WINE.  The program doesn't run.  Why?  Because _most_
Windows applications _expect_ to be able to _write_ to their application
directory.  _Every_ recent (let alone legacy) Windows application
expects this, even Microsoft's own (even in-house MS Application
developers are clueless when it comes to writing applications that are
not security ignorant), as well as as the pre-.NET, Win32 API.  It is a
_massive_ security issue, because the application must have write
access.

Even if the application does not require the "start up" directory to be
the same location as its binaries, it still requires it to be writable. 
Slowly but surely, Windows applications are starting to use the user's
home directory.  But where is that?  It varies from application to
application, because there is no such thing as a "standard user home
directory" in Windows -- _despite_ Microsoft _finally_ standardizing on
"Documents and Settings" as of Windows 98 SE and Windows 2000 on-ward. 
Many programs simply do not use it.

"B" is a follow-on to "A".  Windows has a MAJOR LACK OF STANDARDIZATION
when it comes to its FILESYSTEM HEIRARCHY.  One of my favorite ironies
of "UNIX having MORE STANDARDS than Windows" was when Microsoft was
maintaining both MS-DOS 7.x-based Windows (95, 98, ME) and NT-based
Windows (NT/2000/XP) and they _differed_ in standard locations, right
down to user files.  Binary and temporarily files are all over the
place, let alone standard data file locations.

This is why NTFS fragments so horrendously versus UNIX.  No matter how
well the NT administrator tries to plan his systems, network, etc...
fragmentation issues on Windows platforms occur because of the
applications, as well as a few OS design issues.  That makes it 100x
_worse_ than UNIX.  Furthermore, the included degragmentor with NT 5.x,
a "lite" version of Diskkeeper, does not defragment the directory
structure in NTFS, only the file allocation.

Lastly, Microsoft has not written the NTFS filesystem driver to be more
"pro-active" in allocating contiguous blocks to files.  Heck, the FAT
driver in _all_ versions of Windows was _never_ re-written to do this
(even the Linux MSDOS/VFAT drivers were better at preventing
fragmentation!).  Maybe that will happen with NTFS 6.0 in NT 6.0
(Longhorn), as they become more conscience on performance with the
"embedded SQL" implementation.  Who knows?

- UNIX

Fragmentation is virtually non-existent in UNIX.  Strict separation of
binaries, temporary files and data is easily done.  The applications
_respect_ it, from the standard C libraries of _any_ UNIX platform (even
the smallest) to the way the programs assume _no_ write access to _any_
directory but the user's home directory.  And there is _no_ such thing
as a "start up" directory where a program _requires_ write access
(unlike even the _entire_ Win32 API).

People say the problem with UNIX is applications.  I tell you the
problem with Windows has _always_ been the applications, from the Win32
API (although .NET improves _some_ things) to the _massive_ignorance_ of
good programming of Microsoft's _own_ application division.



From: Roger_Hill@May-Co.com
> A question was posed to me the other day about filesystems.
> Which type of filesystems are better ? Windows NTFS or Linux's ?
> Someone (a Windows NT person) stated that NTFS is a much more
> secure file system, has more control (ACL's) I thought Linux also
> had an ACL function for it's filesystems.
> Thought ?

From: Sean Jewett <sean@rimboy.com>
> Depends on what file system you want to use under Linux, where you
> actually have a choice.  Oh yeah, you can use FAT32 under NT but then you 
> lose the ACL support as I recall.  
> That's relative.  The underlying OS is full of holes.  In addition, those 
> ACL's are not doing a lick of good if the various virii can make critical 
> changes or better yet wipe important parts of the OS or harddrive.  It's 
> nothing an rm -rf * cannot do with root but at least from the user 
> perspective if they don't have root they cannot do too much damage.
> My limited experience with NT ACL's is that they're a pain in the rear and
> don't work as advertised.  I still maintain that Novell has ACL's right
> depsite everything else they have wrong.  However, the Netware FS is
> supposedly being ported to Linux (if they've not announced that yet) and
> that should give Linux the ACL stuff it really needs.  XFS supports ACL's,
> unfortunately I'm not aware of what else may or may not support ACL's.  
> I've not played with ACL's under linux in part because I've not had a
> reason to.  While I'm leery about the NetwareFS after talking with one of
> their engineers they're bringing quite a bit to the table.

From: Robert Citek <rwcitek@alum.calberkeley.org>
> What metric are you using to quantify "better"?
> How are you measuring "more secure"?  Size of ACL's?
> As an anecdote, we had an NT server using NTFS go down about a year 
> ago.  After struggling and failing to get the NT machine up, we booted 
> with Knoppix, mounted the NTFS partition read-only, and were able to 
> copy all of the data over the LAN to a working NT machine.  All those 
> ACLs in NTFS didn't stop us from getting to the data, which in our case 
> was a good thing.
> In short, you cannot evaluate the security of a filesystem without 
> considering the operating system.

From: Steven Pritchard <steve@silug.org>
> As I understand it, NTFS is a log-structured filesystem.  (Like many
> things in NT, that kind of filesystem was the Big Thing to CS
> professors at the time it was designed.)  For a description of what
> that means, see here:
>   http://en.wikipedia.org/wiki/Log_structured_filesystem
> There's also a fairly detailed description of NTFS specifically here:
>   http://en.wikipedia.org/wiki/NTFS
> (I'm getting sucked into Wikipedia now...)
> This page says NTFS is a journalling filesystem:
>   http://en.wikipedia.org/wiki/Journaling_file_system
> So I'm going to stop looking at all this and leave the real answer as
> an exercise for the reader (and Google).
> Anyway, back to log-structured filesystes for a moment...  There have
> been a couple of these for Linux at one point or another, such as
> this:
>   http://www.complang.tuwien.ac.at/czezatke/lfs.html
> (Like I said, it's a big thing for CS people.)
> So, switching gears, "better" is more than a little subjective, even
> ignoring Windows-vs-Linux, so I'm just going to focus on Linux
> filesystems.  The "default" filesystem on Linux is ext2/ext3 is
> relatively primitive by today's (CS :) standards, but it was designed
> to be extended, so it has been slowly incorporating features of more
> advanced filesystems over time.  It is certainly not a bad filesystem,
> and it has the distinct advantage of having a *huge* installed base
> and an *insane* amount of testing for the filesystem itself and the
> tools.
> The truely advanced filesystems available for Linux are XFS and, to a
> lesser extent, JFS.  As I understand it, both support on-line resizing
> out of the box, as well as a lot of other nifty stuff.  Of course,
> both have the advantage of having a long (previous) life on other
> operating systems (IRIX in the case of XFS, and both AIX and OS/2 in
> the case of JFS).  Of the two, XFS almost certainly has the least
> legacy cruft.
> Theoretically reiserfs is up there with XFS and JFS, but in practice
> it appears to be a bit unreliable.  That may just be a function of the
> tools, which have probably had a lot less testing than those for any
> of the other mainstream filesystems.
> I think every Linux filesystem has some kind of ACL support, but
> there's a school of thought that the simpler security provided by
> standard Unix filesystem permissions tend to give much better security
> given that they are easier to understand and easier to verify.
> The real security problem with NTFS isn't the filesystem itself
> though...  The real problem is that M$ ships every version of Windows
> with "full control" enabled for "all" on every filesystem, including
> the one where Windows itself is installed.  The only thing that even
> begins to save Windows is that files that are actively in use by the
> operating system can't be changed.  (Of course, that can be easily
> circumvented by adding things to the registry, modifying things that
> are only used on boot-up, etc.)  And to top it all off, if an
> administrator decides to lock down a Windows box, things are likely to
> break in big ways since most Windows apps (including ones from M$)
> have *no* understanding of how to work with filesystem-level security
> enabled in any way.
> In other words, NTFS could be the most advanced filesystem ever, and
> it wouldn't help Windows security one bit.

From: L. V. Lammert <lvl@omnitec.net>
> *Basic* security is roughly equivalent. NTFS has a lot more
> 'properties' for groups and users, but that can be a plus or minus
> depending on the viewpoint.  Sone of the newer Linux FSs offer many
> improvements over basic NTFS, however I am not up to speed on them.


-- 
Bryan J. Smith, E.I. -- Engineer, Technologist, School Teacher
b.j.smith@ieee.org



-
To unsubscribe, send email to majordomo@silug.org with
"unsubscribe silug-discuss" in the body.