Hard Links vs. Soft Links

Contents

Contents of this writing:

  • Contents
  • Overview
  • Introducing Hard Links
  • Introducing Soft Links
  • Creating a Hard Link
  • Creating a Soft Link

Overview

This writing talks about hard links and soft links; two of the nice features of NTFS file system.

You can divide file links into two categories:

  • Normal Links (shortcuts)
  • Symbolic Links:
    This divided into two subcategories:

    • Hard Links (for files)
    • Soft Links (for directories)

Introducing Hard Links

A hard link is a way to represent a single file (i.e. data volume) by more than one path. In other words, it is a way to create multiple directory entries for a single file.

Consider the following example, we could have two files, e.g. D:DocumentsBills.doc and D:FilesCompanyInvoices.doc, both files represent the same data, so when you change one file the other changes too!

A hard link is almost like a normal link (i.e. shortcut) but there is a big difference. If you delete the source file of a shortcut, the shortcut would be broken (points to a non-existing file.) A hard link on the other hand, would be still working fine if you delete the source file as if it was just a copy.

Notice that, hard disk space is not multiplied for each hard link. Because they are share the same data volume, they share the same size.

There is something that you should be aware of. File attributes are not updated for each of the hard links. This means that if you set a hard link (e.g. D:FilesCompanyInvoices.doc) to read-only, it is the only link that is set to this attribute. Other hard links including the source would be still read-write (unless you set them too.)

Hard links are only supported on NTFS file system; they are not supported by Win9x versions of Windows. Besides this, hard links creation is limited to the same local disk (or network drive) of the source file. For example, you cannot create a hard link on the C: drive for a file on the drive D: or on a network share.

Introducing Soft Links

Soft links (also called junctions,) are identical to hard links except that soft links are designated for directories not files.

For example, if you have one or more soft links to the directory D:Tools (e.g. D:Software and D:ProgramsTools,) all directories (including the source) would be updated when any one of them changes. That is, if you created a new file in one directory, it is created in all directory links, and so on.

Like hard links, soft links require NTFS file system. Plus, deleting the source directory does not affect other directories, all links share the same data volume, changing attributes of one directory does not be applied to others.

Unlike hard links, creating soft links is a special feature of Windows Vista and Windows Server 2008 (and future versions of course.) Therefore, it cannot be applied to earlier versions of Windows (like Windows XP or Windows Server 2003.)

Creating a Hard Link

Unfortunately, neither hard links nor soft links are supported by the .NET Framework. Therefore, you will need to dig into the Windows API to allow your application to consume this feature.

You can create a hard link using a single line of code using a simple call to the Win32 function, CreateHardLink(), that resides in the Kernel32.dll library. The definition of this function is as follows:

BOOL CreateHardLink(
    LPCTSTR lpFileName,
    LPCTSTR lpExistingFileName,
    LPSECURITY_ATTRIBUTES lpSecurityAttributes
    );

This function accepts three arguments;

  • lpFileName:
    The path of the new hard link to be created. Notice that, it should be resides in the location (local disk or network share) as the source file.
  • lpExistingFileName:
    The path of the source file to create the hard link from.
  • lpSecurityAttributes:
    The security descriptor for the new file. Till now, it is reserved for future use and it should not be used.

This function returns TRUE if succeeded or FALSE otherwise.

Keeping the definition of the function in our minds, we can create its PInvoke method in C#. Simply, this is our C# PInvoke method:

[DllImport("Kernel32.dll", CharSet = CharSet.Unicode )]
static extern bool CreateHardLink(
    string lpFileName,
    string lpExistingFileName,
    IntPtr lpSecurityAttributes
    );

As you know, we have set the character encoding to Unicode because Windows NT allows Unicode file names. In addition, we have marshaled lpSecurityAttributes as System.IntPtr to allow passing it a null value using System.IntPtr.Zero.

Then, you simply call the function:

Change the source file to meet an existing file into your system. In addition, check that the directory of the new link exists or the call would fail.

Take into consideration that the function creates the new file for you, it should not be found or the function would fail (as the case with soft links.)

CreateHardLink(
    "D:\Files\Company\Invoices.doc",	// New Link
    "D:\Documents\Bills.doc",			// Source
    IntPtr.Zero);

And this is another link:

CreateHardLink(
    "D:\Important\Work Invoices.doc",			// New Link
    "D:\Files\Company\Invoices.doc",			// Source
    IntPtr.Zero);

As you see, it does not matter if you created the new link from the original file or from another link.

Now try to change any of the links, and check the others. They are all updated at the time you commit the change (save the file, for instance.)

Creating a Soft Link

Like hard links, you can create a soft link (a junction) using a single call to a Win32 API function; it is CreateSymbolicLink() function. Again, this function is only supported by Windows Vista and Windows Server 2008 (and future versions.)

Actually, the function CreateSymbolicLink() can be used to create both types of symbolic links, hard and soft links. The definition of this function is as follows:

BOOLEAN WINAPI CreateSymbolicLink(
    LPTSTR lpSymlinkFileName,
    LPTSTR lpTargetFileName,
    DWORD dwFlags
    );

This function accepts three arguments, the first and the second are identical to the CreateHardLink() function:

  • lpSymlinkFileName:
    The path of the new link (file or directory) that resides in the same drive (local disk or network share) of the target file.
  • lpTargetFileName:
    The path of the existing source (file or directory.)
  • dwFlags:
    The type of symbolic link. It can take the following values:

    • 0x0:
      The link is a hard link (source and destination are files.)
    • 0x1 = SYMBOLIC_LINK_FLAG_DIRECTORY:
      The link is a soft link (source and destination are directories.)

The following is the PInvoke method for our function:

[DllImport("Kernel32.dll", CharSet = CharSet.Unicode)]
static extern bool CreateSymbolicLink(
    string lpSymlinkFileName,
    string lpTargetFileName,
    uint dwFlags
    );

const uint SYMBLOC_LINK_FLAG_FILE		= 0x0;
const uint SYMBLOC_LINK_FLAG_DIRECTORY 	= 0x1;

The following code creates two soft links for one directory:

Again, the source should be the only directory that exists in your machine or the function would fail.

CreateSymbolicLink(
    "D:\Software",         // New Link
    "D:\Tools",            // Source
    SYMBOLIC_LINK_FLAG_DIRECTORY);

// Source
CreateSymbolicLink(
    "D:\Programs\Tools",  // New Link
    "D:\Software",         // Source
    SYMBOLIC_LINK_FLAG_DIRECTORY);

Now try changing one directory, other links change too. Try deleting the source, nothing would be happen to the other links.

Programmatically Compress and Decompress Files

هذه المقالة متوفرة أيضا باللغة العربية، اقرأها هنا.

This lesson is very easy. This lesson focuses on how to compress and decompress files programmatically using .NET Framework and C# -or any language that supports .NET of course.

Currently .NET Framework supports two types of compression algorithms:

  • Deflate
    This is a very simple algorithm, used for simple compression and decompression operations. Don’t use this algorithm except if you intend to use compressed data only in your application because this algorithm is not compatible with any compression tool.
  • GZIP
    This algorithm uses internally the Deflate algorithm. This algorithm includes a cyclic redundancy check value for detecting data corruption. Also it’s compatible with most compression tools because it writes headers to the compressed file, so compression tools -like WinZip and WinRAR- can easily access the compressed file and decompress it as well. This algorithm also can be extended to use other algorithms internally other than Deflate.

For a complete GZIP reference see RFC 1952 (GZIP File Format Specification).

The most powerful compression tool now is WinRAR.

Fortunately, whether using Deflate or GZIP in .NET, code is the same; the only thing that needs to change is the class name.

Deflate and GZIP can be found in the namespace System.IO.Compression that resides on assembly System.dll. This namespace contains only three types, the two algorithms implementation classes DeflateStream and GZipStream -both inherit directly from System.IO.Stream class-, and an enumeration CompressionMode that defines the operation (compression or decompression).

Compressing a file

The code for compression is very simple:

Code snippets in this lesson rely on the fact that you added two using (Imports in VB) statements, System.IO and System.IO.Compression.

// C# Code

string fileToBeCompressed = "D:\My Great Word Document.doc";
string zipFilename = "D:\CompressedGreatDocument.zip";

using (FileStream target = new FileStream(zipFilename, FileMode.Create, FileAccess.Write))
using (GZipStream alg = new GZipStream(target, CompressionMode.Compress))
{
    byte[] data = File.ReadAllBytes(fileToBeCompressed);
    alg.Write(data, 0, data.Length);
    alg.Flush();
}

Code explanation:
If you are going to compress a file then you must specify the CompressionMode.Compress option, and also you must specify a Stream that will the data be written to.
After creating the class you can compress the data by using its Write() method.

If you intended to use the Deflate algorithm, just change the class name to DeflateStream.

Decompressing a file

The code that decompresses a compressed file is very similar:

// C# Code

string compressedFile = "D:\CompressedGreatDocument.zip";
string originalFileName = "D:\My Great Word Document.doc";

using (FileStream zipFile = new FileStream(compressedFile, FileMode.Open, FileAccess.Read))
using (FileStream originalFile = new FileStream(originalFileName, FileMode.Create, FileAccess.Write))
using (GZipStream alg = new GZipStream(zipFile, CompressionMode.Decompress))
{
    while (true)
    {
        // Reading 100bytes by 100bytes
        byte[] buffer = new byte[100];
        // The Read() method returns the number of bytes read
        int bytesRead = alg.Read(buffer, 0, buffer.Length);

        originalFile.Write(buffer, 0, returnedBytes);

        if (bytesRead != buffer.Length)
            break;
    }
}
' VB.NET Code

Dim compressedFile As String = "D:CompressedGreatDocument.zip"
Dim originalFileName As String = "D:My Great Word Document.doc"

Dim zipFile As New FileStream(compressedFile, FileMode.Open, FileAccess.Read)
Dim originalFile As New FileStream(originalFileName, FileMode.Create, FileAccess.Write)
Dim alg As New GZipStream(zipFile, CompressionMode.Decompress)

While (True)
    ' Reading 100bytes by 100bytes
    Dim buffer(100) As Byte

    ' The Read() method returns the number of bytes read
    Dim bytesRead As Integer = alg.Read(buffer, 0, buffer.Length)

    originalFile.Write(buffer, 0, bytesRead)

    If (bytesRead  buffer.Length) Then
        Exit While
    End If
End While

Code explanation:

First, we create a file stream to read the ZIP file, and then we created our algorithm class that encapsulates the file stream for decompressing it.

Then we created a file stream for the original file, for writing the decompressed data.
After that, we read the data compressed 100bytes after each other -you may prefer more- and write this data to the original file stream.

By encapsulating disposable objects into using statements you become assured that every object will be disposed in a certain point -end of the using statement- and no object will retain in memory.

Try it yourself

Develop a simple application that can be used to compress and decompress files.

This application can compress several files into one file and decompress them as well.

Besides algorithms classes you may use System.IO.BinaryWriter and System.IO.BinaryReader to get full control of how the file contents will be.

Creating Transacted Files

Lastly but not last, and after a long while, Windows Vista introduced a way to create transacted files or even to write to registry.

While market grows remarkably in the last years, its requirements increase as well. And every day you face a new problem that you must overcome to accommodate market requirements.

Transacted operations are one of the commonly demanded requirements by market.

While it’s true that you can easily do database operations transactionally, it’s not enough. Sometimes you will need to write files or make changes to registry in a transactional way.

With previous versions of Windows, you weren’t able to create a file transactionally without writing a huge amount of disorganized code to create it manually. You couldn’t use normal transactions or even COM+ Enterprise Services to create this type of transactions.

With Windows Vista you can easily create transacted files the common way you used to create normal files.

Unfortunately, from .NET 3.5 you cannot create transacted files. So you need to dive into API to create it.

Not Windows Vista only that supports this type of transactions, Windows Server 2008 also supports it.

Creating a transacted file function

To create a transacted file you can use the Kernel23.dll new function CreateFileTransacted.

This function takes the same arguments as the normal CreateFile function plus three more arguments that we are interested of the first one of them, the transaction handle that will be used while creating the file and writing to it.

The definition of this function in C is as follows:

HANDLE CreateFileTransacted(
    LPCTSTR lpFileName,
    DWORD dwDesiredAccess,
    DWORD dwShareMode,
    LPSECURITY_ATTRIBUTES lpSecurityAttributes,
    DWORD dwCreationDisposition,
    DWORD dwFlagsAndAttributes,
    HANDLE hTemplateFile,
    HANDLE hTransaction,
    PUSHORT pusMiniVersion,
    PVOID pExtendedParameter);

We are interested in 6 parameters of these 9.

  • lpFileName:
    The path of the file being created.
  • dwDesiredAccess:
    Defines how file will be accessed. Read only (0x80000000), right only (0x40000000), or both read and write (0xC0000000).
  • dwShareMode:
    Defines the sharing mode -during the operation- for the file. Enabling reading file contents (0x1), writing to it (0x2), reading and writing (0x3), or deleting the file (0x4).
  • dwCreationDisposition:
    Action to take if file exist or do not exist. This argument can take one of the values:
    0x1: Create the file and throw an error if it exists.
    0x2: Always create the file even if it exists.
    0x3: Open the file and throw an error if it doesn’t exist.
    0x4: Open the file or create it if it doesn’t exist.
    0x5: Open the file and clear its contents. Or throw an error if it doesn’t exist.
  • dwFlagsAndAttributes:
    Any additional options. This can be a combination of file attributes like Archive, Hidden, and Encrypted. Also it supports combining options like deleting the file after closing it.
  • hTransaction:
    A handle to our created transaction that we wish to use it in this operation. To get the handle you may take a step further into COM.

Also there’re three arguments that we are not interested on, so you can safely pass it a NULL value:

  • lpSecurityAttributes:
    A SECURITY_ATTRIBUTES object contains an optional security descriptor information for the file.
  • hTemplateFile:
    A handle to a file to read it’s attributes and options to fill the arguments automatically.
  • pusMiniVersion:
    The miniversion to be opened. When creating transacted file -by specifying the hTransaction argument, this must be NULL.
  • pExtendedParameter:
    Reserved.

For a complete discussion of this API function, see MSDN Library.

PInvoking CreateFileTransacted API function

PInvoke is a service that enables you to call unmanaged functions in DLLs, such as those in the Win32 API like the CreateFileTransacted API function last mentioned.

PInvoke stands for Platform Invokation.

In order to PInvoke a function you need to know some things:

  1. Where this function resides (which DLL).
  2. Function definition and order of arguments -if found.-
  3. How to marshal between unmanaged types in .NET types.

Marshaling in .NET is the process of creating a bridge between new .NET types and legacy COM types. This is done by finding equivalents for those legacy types in the new data types or creating it if needed.

Most of the COM data types have equivalents in .NET, also it’s very easy to create your own.

For the function on hands we could write it in C# as following:

[DllImport("kernel32.dll")]
    public static extern
    IntPtr CreateFileTransacted(
    string lpFileName,
    uint dwDesiredAccess,
    uint dwShareMode,
    IntPtr lpSecurityAttributes,
    uint dwCreationDisposition,
    uint dwFlagsAndAttributes,
    IntPtr hTemplateFile,
    IntPtr hHandle,
    IntPtr pusMiniVersion,
    IntPtr pExtendedParameter);

Code explanation:
static extern modifiers are required for creating PInvoke functions.
DllImport attribute used to define the DLL that contains the function.
System.IntPtr is the managed type equivalent to unmanaged HANDLE.
LPCSTR can be easily replaced by the managed System.String.
In unmanaged code DWORD is a 32-bit unsigned integer, so we could safely replace it with System.UInt32.

Because we need to pass a NULL value to the LPSECURITY_ATTRIBUTES argument we marshaled it as IntPtr, so we can use IntPtr.Zero to pass NULL values.
We did that too with the last two arguments.

For creating the PInvoke methods easily, you could use the Red Gate’s PInvoke.NET Visual Studio add-in.
Also you should visit the interop wiki.

After we create our PInvoke method we can call it. But after that we need to create the transaction and get its handle to fill the arguments of the method.

To get a transaction handle we need to dive into COM and call a method to get the handle. This method is wrapped to .NET using COM Interop.

[ComImport]
[Guid("79427A2B-F895-40e0-BE79-B57DC82ED231")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IKernelTransaction
{ void GetHandle(out IntPtr ktmHandle); }

Don’t forget adding a using System.Runtime.InteropServices statement.

Now we can call the method:

private const uint FILE_ATTRIBUTE_NORMAL = 0x80;
private const uint GENERIC_WRITE = 0x40000000;
private const uint CREATE_ALWAYS = 0x2;
private const uint FILE_SHARE_NONE = 0x0;

static void Main()
{
    using (TransactionScope scope = new TransactionScope())
    using (FileStream stream = CreateTransactedFile("D:SampleFile.txt"))
    using (StreamWriter writer = new StreamWriter(stream))
    {
        writer.WriteLine("Hello, World!");

        // To commit the transaction use the followind line
        scope.Complete();
    }
}

public static FileStream CreateTransactedFile(string fileName)
{
    IKernelTransaction ktx = (IKernelTransaction)
        TransactionInterop.GetDtcTransaction(Transaction.Current);

    IntPtr txHandle;
    ktx.GetHandle(out txHandle);

    IntPtr fileHandle =
        CreateFileTransacted(fileName,
        GENERIC_WRITE, FILE_SHARE_NONE, IntPtr.Zero,
        CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL,
        IntPtr.Zero, txHandle, IntPtr.Zero, IntPtr.Zero);

    return new FileStream(fileHandle, FileAccess.Write);
}

Don’t forget adding a reference to System.Transactions and a using statement.

Code explanation:
Constants define the input for the function arguments.
In method CreateTransactedFile() we get the handle of the ambient transaction created in method Main(). Then we call the CreateFileTransacted() function which returns a pointer to the create file. Lastly we created the FileStream object with the created the file handle and used it to create a StreamWriter object to write textual data to the file.

Calling Commit() of the transaction object ensures that all operations completed successfully and now we can apply it. Otherwise, nothing would be done. That’s the main characteristic of transactions. All or nothing.

You don’t need to close the transaction handle manually because when disposing the TransactionScope object it will close its handle automatically.
If you are interested in closing it manually you can use the .NET wrapper System.Runtime.InteropServices.Marshal.Release() method. Or try the difficult way using the CloseHandle unmanaged handle.

Transactional support not limited to creating a file only. Most file system operation now support transactions, some examples are CopyFileTransacted, CreateDirectoryTransacted, CreateHardLinkTransacted, DeleteFileTransacted, MoveFileTransacted, RemoveDirectoryTransacted, and SetFileAttributesTransacted.

The complete example can be downloaded here.