Providing integrity for Encrypted data with HMACs in .NET

Note: This post is a continuation of my previous article on AES Crypto in .NET

Once again I’m about to dip into and scratch the surface of Cryptography. So here’s the disclamer: This is not my job. I don’t do this for a living. Don’t ever make up your own encryption algorithms. Try not to write your own cryptographic code. Don’t take anything that I say as the advice of a security expert or legal advice or assume that any code in this post or any linked posts are correct in any way shape or form. If you have a project that requires cryptographic security, I suggest you find someone who has been doing this far longer than I have to write the code for you. Then have several 3rd party security firms review or write your security code. In short, I’m not responsible for your mistakes, or the correctness of the code or writing presented here.

As even more of a taste of just how hard it is, I admit that I thought the last article I wrote covered most of the basics of modern encryption in .NET pretty well. The result? Nope. Missed something big. See, cryptography is hard not because it’s hard to say “take a password plus this data and make it so someone can’t see the data without the password”, but because of all the ways someone could get access to your password, or access your data when its not encrypted, or access your computer when it’s unlocked, or crack your password if you pick a bad password to begin with. Take a look at this reddit thread article that nicely rips apart my previous article. In all seriousness, I really appreciate the feedback, especially when it comes to security because it’s hard, and the devil is in the details.

I forgot to cover data integrity in my last article. Encryption is not enough. Redditors referred to it as authentication, which is one of the uses of HMAC’s, but authenticity is not whats important in the example case I presented for my original article. We are interested in “How to detect that someone or something accidentally or maliciously changed the encrypted data”, essentially, we’re interested in cryptographic data integrity. In addition to that there were a few other things wrong with the article that the redditors pointed out, such as my salt values being overly large, just picking CBC without covering other chaining methods, not discussing how to remove keys from memory, and referencing Jeff Atwood. The long and the short of it, they’re completely right. Getting crypto correct is hard, and good crypto systems are worth millions of dollars. You’re probably not getting paid that much to write a little encryption. So use a library thats already been written and provides high level abstractions and don’t write crypto code yourself.

Alright. HMAC, what is it? The MAC stands for a Message Authentication Code, the H stands for Hash. Put it all together and you have a Hash-based Message Authentication Code. It’s a hashing function thats deliberately designed to resist malicious tampering. The key is preventing malicious tampering. A normal hash function (e.g. MD5 or SHA1) would detect accidental byte tweaks, but somebody maliciously tampering with your data could tinker with it, create a new hash code for the data, replace the old hash code, and you would never be the wiser. For this reason, Message Authentication Codes generally fall into the category of keyed hash algorithms since they use a key or derived key and mix it with a hashing or encryption function to produce a value that an attacker can’t reproduce, providing both data integrity, and (if you happen to be in a sender and receiver role where both parties share a key) authentication. There are however, a couple of things to take into account. First, we shouldn’t use the same key twice to encrypt and hash the data. The more times the same key is used, especially against a known piece of data, the more likely an attack can be developed and used to figure out our key. Arguably, the final key used for encryption and the final key used for the message authentication should be different, as different as possible. The best way to do this would be to append or change the key such that both the encryption key and authentication key run through the KDF (Key Derivation Function) from different starting points. As an example, consider the following:

  1. // Derive the passkey from a hash of the passwordBytes plus salt with the number of hashing rounds.
  2. var deriveKey = new Rfc2898DeriveBytes(password, passwordSalt, 10000);
  3. var deriveHMAC = new Rfc2898DeriveBytes(password, hmacSalt, 10000);
  1. // This gives us a derived byte key from our passwordBytes.
  2. var aes256Key = deriveKey.GetBytes(32);
  3. var hmacKey = deriveHMAC.GetBytes(32);

Because the hash function mixes the password with the salt, and because we have different salts, after only one round the derived keys will already be different on account of the salt value. So we have a derived key. One for the actual encryption, one to prevent tampering and provide data integrity.

On a side note, it’s arguably better to authenticate the encrypted output (encrypt then authenticate) rather than authenticate the plaintext, then encrypt, or authenticate and encrypt. Again, I know it’s beating a dead horse, but cryptography is hard, and ultimately, the security of a system is going to depend on the security of the entire system, not just the individual parts. So. We have an HMAC key, we have an encryption key, and we know that we want to encrypt, then authenticate the encrypted output. In addition, we want to make sure anything else that could easily be tampered with is also authenticated, such as our Initialization Vector, since any change to it can easily affect our decrypted output. Another small advantage thats almost not worth mentioning is that by authenticating the encrypted output instead of the plain text is that we can detect if anything has changed even before we start decrypting the text. So here’s an example. In this case, I chose to use HMACSHA1, there’s others on the MSDN but I chose this particular one since it uses the same hash algorithm used internally by the KDF I used in my previous post, Rfc2898DeriveBytes, aka (PKDF2).

  1. var hmac = new HMACSHA1(hmacKey);
  2. var ivPlusEncryptedText = iv.Concat(cipherTextBytes).ToArray();
  3. var hmacHash = hmac.ComputeHash(ivPlusEncryptedText);

In this case, we’re using our derived hmacKey, and we’re computing the hash of both the initialization vector concated with our encrypted ciphertext. That gives us everything we need to have a self validating “package” of data that is secured and can’t be tampered without us knowing unless the attacker knows our key or can break AES256 encryption, but at that point this whole discussion is pointless.

With decryption, remember how I said we compute the Encryption and HMAC key separately? If we did that, and if we computed the hmac over the encrypted data, we can perform the validation step on the data before we compute our decryption key. The only reason we would do this is so that if the data is invalid or has been tampered with we don’t take the time to also compute the decryption key. Small things, but I wanted to explain why the key computation for the encryption and hmac is kept separate:

  1. var deriveHmac = new Rfc2898DeriveBytes(password, hmacSalt, 10000);
  2. var hmacKey= deriveHmac.GetBytes(32);
  3. var hmacsha1 = new HMACSHA1(hmacKey);
  4. var ivPlusEncryptedText = ivBytes.Concat(encryptedBytes).ToArray();
  5. var hash = hmacsha1.ComputeHash(ivPlusEncryptedText);
  7. if (!BytesAreEqual(hash, hmac))
  8. throw new CryptographicException( "Your encrypted data was tampered with!" );
  10. var deriveKey = new Rfc2898DeriveBytes(password, passwordSalt, 10000);
  11. var aes256Key = deriveKey.GetBytes(32);
  13. using (var transform = new AesManaged())
  14. {
  15. using (var ms = new MemoryStream(encryptedBytes))
  16. {
  17. using (var cryptoStream = new CryptoStream(ms, transform.CreateDecryptor(aes256Key, ivBytes), CryptoStreamMode.Read))
  18. {
  19. var decryptedBytes = new byte[encryptedBytes.Length];
  20. var length = cryptoStream.Read(decryptedBytes, 0, decryptedBytes.Length);
  22. var decryptedData = decryptedBytes.Take(length).ToArray();
  23. }
  24. }
  25. }

So, there you go. Basic explanation about why HMAC’s are important, what I missed, some code, and the disclaimer to write security code at your own risk. Full demo code, demo code output, and a bunch of random links after the break.

See the code, output, and related links

More than you ever wanted to know about AES Crypto in .NET

First, before I begin anything, I want to point out that cryptography is hard. REALLY hard. There are so many possible points of failure, and those points of failure and methods of attack can change depending on what the purpose of encrypting the data is. If you do it wrong, you get lawsuits, and/or you end up on the front pages of major news organizations for data security breaches. Or a rival nation gets your top-secret plans to rule the world. You get the idea. I’ll touch on some of the technical points of failure briefly; I’m mainly going to be exploring this topic through a fairly simple scenario: A password vault file, encrypted with a password, and stored locally on my computer (potentially synced to other computers through some syncing service).

As with any good crypto, the first line of defense is always preventing an attacker from gaining access to the encrypted data in the first place. If data is going over a network, encryption should be part of that as well, and that brings in a whole new host of issues that are outside of the scope of this article. Since I’m planning on storing the data locally on my hard drive, the first line of defense is me and my computer: Lock my computer when I walk away. Make sure there’s no spyware or malware on my computer, no keyloggers or other devices that could steal my password, etc… Going back to my original point about crypto being hard, it’s hard not only because writing the code is hard, it’s difficult because of all the ancillary ways someone could potentially get access to my data that I have to account for.

If I leave my computer unlocked, someone could steal data off of it. Install a keylogger. If I leave this password app open someone could just go in and look at my passwords after I’ve decrypted them. If they have physical access someone could install a HARDWARE keylogger between my keyboard and computer. Pull out my hard drive and clone it. Hack into the company Wi-Fi network and remote access my computer. Or even STEAL my computer. Or TSA could confiscate it. And/or force / legally compel me to reveal my password. You get the idea. There is no such thing as absolute security or secrecy.

However, let’s assume for a moment that an attacker DOES somehow manage to get access to my vault file. (Maybe by getting access to my computer when the vault program is closed and the data encrypted, or a folder syncing program has a hiccup and spams a public link to it on the Internet) My password file is in the hands of someone actively trying to get at my data, and the only thing protecting the data is my password and good cryptography. So. How do I use good cryptographic practices to secure my data?

Since we want to use the latest and greatest, we pick AES (Advanced Encryption Standard, successor to Date Encryption Standard and Triple Data Encryption Standard or Triple DES), Rijndael 256 to be specific (It’s good enough for government work…). In the System.Security.Cryptography namespace, Microsoft has kindly supplied us with the RijndaelManaged class and a pre-provided implementation of the AES standard. I ALWAYS recommend using a pre provided class instead of attempting to roll your own. The ones from Microsoft have gone through EXTENSIVE testing, code, and security review to verify the correctness of the algorithms and code. Frankly, if you write your own, yours will probably have bugs. Bad ones. Bugs that might get you put on the front page of major news organizations for data security breaches if you do it wrong. Warning given, point made, moving on.

Now, there are a couple of things you’ll need in order to do symmetric encryption when we crack open the RijndaelManaged class and try to create an encryption transform:

  • A Key
  • An Initialization Vector or IV
  • Your Data

The Key

First, the key, which is represented as a byte array of some fixed size. The size of your key is important, the number of bits has to match what the algorithm supports, in the case of Rijndael, it supports key sizes of 128, 192, and 256. In our case, we’ll be using a key size of 256. The number of bytes we need is 256 divided by 8 (remember, the key size for the algorithm is specified in bits, but our code mainly works with bytes). So if I’m declaring a byte array for the key for AES 256, it would be new byte[32].

Derive your Key

Now wait, how the heck to I remember a 32 number combination? I can’t. You probably can’t, and unless you’re a brainiac with an eidetic memory I wouldn’t suggest trying with any reasonable hope of success. So, since I have now come to this sad conclusion about my memory, I need some way to take a password or key that I can memorize and turn it into a 32 byte key. Fortunately, there’s a way to do just that by way of a password based key derivation function called PBKDF2. It’s also a standard (RFC 2898, which is only important because the code uses the name of the spec for the class name and not PBKDF2 like I would expect… go figure). Essentially, PBKDF2 is a piece of code that hashes an arbitrary length of text into a pseudo random key. Now, we could use something like MD5 to do something similar, but PBKDF2is designed with cryptography in mind, and to be slow on purpose. Why does it need to be slow? Simply this, cryptography is a big huge lever that makes it easy to go one way, and hard to go another. MD5 is FAST to compute, and someone trying to break into your encrypted data wants to be able to test as many passwords as possible as fast as possible. PBKDF2is slow and more difficult to compute, and the harder it is, the longer it takes to generate a hash, which dramatically reduces the number of passwords a hacker trying to attack your vault can try per second. (There’s a really good article by Jeff Atwood here: on hashing algorithms in security, rainbow tables, speed hashing, why it’s important to have a slow hash for security, and so on. If you’re interested on reading more about it)

How do we create this hashed key with PBKDF2? The constructor takes 3 things, a string password, an iteration count, and a byte array for something called a salt (Not in that order). The Password is easy, it’s a string or byte array and it’s whatever you choose to use as your “uB3rAw3$omeP@assw0rd!“. iteration count is pretty self-explanatory too, it defaults to 1000, and the bigger the number, the more calculation rounds it does and the longer it takes to calculate the hash. But what about this salt thing? What is it, and why is it important? Do I even care? So here we go.

Salt your Vault

If you’ve heard about salt or salting passwords before, it was probably in reference to a website or service, usually some piece of code that you controlled that sat between your users (or a supposed attacker) and the hashed passwords in your database so that an attacker couldn’t just *make up* a password like zn3lzl that would hash to the same value as the users password like mycoolpassword2 and allow them to log in. If we assume that the attacker has access to your password vault, we can probably assume he has access to your program as well, and it won’t be very hard to figure out some static salt value you’ve stored in your code. So it’s useless right? In some ways yes, in some ways no. IF it’s JUST you, or the attacker is targeting JUST YOUR vault, then yes, it doesn’t matter. However, if for some reason your password vault app becomes popular, and an attacker gains access to, lets say 5000 different peoples vault files, it’s much much easier to check the same passwords across ALL the vaults at once if the salt values are the same. That is again assuming he is not targeting just one particular vault. If they’re all different, the time required to check a password across all the vaults goes up by a factor of 5000, making it less viable to quickly crack multiple vaults. My thought is to simply store the salt along with the vault, since if an attacker has access to your vault, he probably has access to wherever I’d store the salt anyways, and it at least requires him to compute a separate hash for each vault. So, I’ll generate and store a separate, cryptographically random generated salt with my vault file.

Something like this:

  1. var salt = new byte[256];
  3. using (var rnd = RandomNumberGenerator.Create())
  4. {
  5. rnd.GetBytes(salt);
  6. }

As an aside, realize that no amount of encryption can save your data from a bad password. Good hackers are GOING to use huge word lists and fantastic substitution / transposition / combination rules before they even attempt to use a brute force attack. ‘MyL1ttl3P0ny‘ is not a good password. ‘abc123‘ is arguably much worse; but then you’re probably not reading this if abc123 is something you’d use for a password.

Now that we have our password, the salt (loaded from our vault file) and the number of iterations, we can derive a key with the correct size:

  1. var deriveBytes = new Rfc2898DeriveBytes(myPassword, mySalt, 10000);
  2. var aes256Key = deriveBytes.GetBytes(32);

Tada! magic. We now have a byte key from the password that we supplied earlier. Ok, now that we have our key, we can see that when you try to create our encryption or decryption transform that it is requiring something called an initialization vector (IV). I know when I was going through all this that I was thinking “what the heck is an initialization vector?” Since I had to teach myself what it was and why it was important, I’m going to assume somebody also doesn’t know and brain dump what I’ve learned on you all as well.

Block Ciphers, Chaining, and Initialization Vectors

First, we have to understand a few things, how a block ciphers works, how cipher block chaining works, how the IV plays in, and why it’s all important. To begin with, block ciphers. Most computer ciphers these days are done using cipher blocks of a given bit size, usually in round multiples of 2, for example 128 bit or 16 bytes chunks (this is actually the size AES uses). Our clear text gets chunked up into these small, manageable blocks of data. Then encryption transform is then run multiple times on each block, and the resulting blocks are all concatenated together to form the final encrypted output.

Because encrypted blocks are computed independently, changes somewhere in the original unencrypted data might only change the encrypted data for a few blocks of the resulting encrypted output. Furthermore, if you happened to have the same 16 byte chunk repeated again such that it aligned with a different blocks, you will get the exact same block of encrypted output. On one hand, this could be useful if you want to send delta updates to an encrypted file, it also means that attacks against individual blocks are feasibly be easier or useful information could be obtained by comparing one version of the vault to another or even patterns might still exist even within the encrypted date. A really awesome example of why it’s important to apply some additional steps during the encryption process with block ciphers are these three images below: (Courtesy of Wikipedia):

Original | No Block Chaining | Proper Cipher Block Chaining

This is where CBC or cipher block chaining comes in. CBC fixes this problem by taking the previously encrypted block and performing a bitwise XOR between it and the raw bytes of the plain text of the next block to be encrypted. Effectively, this means that a one byte change in the first block will change EVERY OTHER block in the resulting ciphertext. However, there’s one last piece. If the first couple of blocks are NOT changed in any way, you can still leek some information about changes that were made by way of the first changed block. So if I have blocks A B C D E and block C gets changed, CBC will cause the resulting ciphertext for blocks D and E to change. It will NOT affect blocks A and B. So, at last, we finally get to the purpose of the initialization vector. The IV is essentially a completely random block to start off the cipher chain. Makes sense now doesn’t it? If I have a completely random block that I create and pass along, it guarantees that even if the plain text doesn’t change at all, the ciphertext will be completely different each time, assuming I generate a new IV each time I change the cipher text. In practice, encrypted data should be statistically indistinguishable from random noise.

Illustration of the process of a block cipher with CBC for Encryption:

Illustration of the process of a block cipher with CBC for Decryption:

Now that we know why we need an initialization vector, we also know, or can guess, what size it should be without having to look at the documentation (16 bytes, because that’s the block size for AES, and our IV is essentially a known random starting block). Since we should generate a new block and also store it with our vault, I’m going to declare our IV and initialize it the same way we initialized our salt:

  1. var iv = new byte[16];
  3. using (var rnd = RandomNumberGenerator.Create())
  4. {
  5. rnd.GetBytes(mySalt);
  6. rnd.GetBytes(iv);
  7. }

The CryptoStream

Now that we have all the pieces we can put it all together and encrypt our data to a MemoryStream (the memory stream could be anything, including a FileStream, but this is easier for demo and console output):

  1. using(var ms = new MemoryStream())
  2. {
  3. using (var cryptoStream = new CryptoStream(ms, transform.CreateEncryptor(aes256Key, iv), CryptoStreamMode.Write))
  4. {
  5. cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Count());
  6. // cryptoStream.FlushFinalBlock();
  7. }
  9. cipherTextBytes = ms.ToArray();
  10. }

EDIT: Thanks to Nick (see comments), he pointed out that cryptoStream.FlushFinalBlock() is unnecessary if you dispose the cryptoStream before you read the data out from the underlying stream. (This happens automatically with a using statement)

And now the code snippet for decrypting the cypherTextBytes:

  1. using (var ms = new MemoryStream(cipherTextBytes))
  2. {
  3. using (var cryptoStream = new CryptoStream(ms, transform.CreateDecryptor(aes256Key, iv), CryptoStreamMode.Read))
  4. {
  5. var decryptedBytes = new byte[cipherTextBytes.Length];
  6. var length = cryptoStream.Read(decryptedBytes, 0, decryptedBytes.Length);
  8. var decryptedText = Encoding.UTF8.GetString(decryptedBytes.Take(length).ToArray());
  9. }
  10. }

Padding under the covers

Some final notes block cipher padding: You’ll notice that when I’m reading the stream I have the following line of code: decryptedBytes.Take(length).ToArray() When you’re using a block cipher like AES, just like the key has to be exactly a certain size, each block of initial data ALSO has to be a certain size. This means you ARE going to get some extra data tacked on the end of your stream that you weren’t anticipating. There’s a couple of ways this can be handled, RijndaelManaged has a couple of padding modes that it can use ranging from filling all the additional slots of data with zeros, or filling them with completely random data, but by default it uses PaddingMode.PKCS7 which fills each extra bytes needed to make the length of the data an even multiple of 16 bytes with the number that represents the number of padded bytes added to fill the empty space. If you have the exact amount of data to exactly to fill the number of blocks, the padding algorithm will add an extra block to ensure that the last byte in the last block that is read represents the amount of padding. Otherwise, depending on whatever data you’re storing, you could accidentally lose some of your data if it was misinterpreted as padding. The crypto stream is aware of the padding and will return the correct length of the original data on the last read call. I simplified the stream reading process for simplicity of demonstrating the use of the crypto stream and how it handles padding, it just reads everything and trims the result with decryptedBytes.Take(length).ToArray(). In real life, you should use or make a ‘real’ stream reader that reads data out of the stream in chunks and aggregates them together, or just serialize / deserialize your objects directly to and from the crypto stream.


The Full Demo

  1. using System;
  2. using System.IO;
  3. using System.Linq;
  4. using System.Security.Cryptography;
  5. using System.Text;
  7. public class CryptoDemo
  8. {
  9. public static void Main(string[] args)
  10. {
  11. TestEncryptionAndDecryption();
  13. Console.ReadLine();
  14. }
  16. public static void TestEncryptionAndDecryption()
  17. {
  18. const string myPassword = "uB3rAw3$omeP@assw0rd!";
  19. const string myData = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. " +
  20. "Morbi rutrum pulvinar purus, nec ornare neque cursus id. " +
  21. "Nunc non tortor est. Morbi laoreet commodo tellus, et suscipit neque elementum eu. " +
  22. "Sed velit lorem, ultricies id varius vitae, eleifend eget massa. " +
  23. "Curabitur dignissim eleifend quam, sit amet interdum velit rutrum vel. " +
  24. "Nulla nec enim tortor.";
  25. var plainTextBytes = Encoding.UTF8.GetBytes(myData);
  27. Console.WriteLine("Password ({0} bytes): ", Encoding.UTF8.GetBytes(myPassword).Length);
  28. Console.WriteLine(myPassword);
  29. Console.WriteLine();
  31. Console.WriteLine("Plain Text ({0} bytes): ", plainTextBytes.Length);
  32. Console.WriteLine(myData);
  33. Console.WriteLine();
  35. var mySalt = new byte[256];
  36. var iv = new byte[16];
  38. using (var rnd = RandomNumberGenerator.Create())
  39. {
  40. rnd.GetBytes(mySalt);
  41. rnd.GetBytes(iv);
  42. }
  44. var transform = new RijndaelManaged();
  46. Console.WriteLine("Salt ({0} bytes): ", mySalt.Length);
  47. Console.WriteLine(Convert.ToBase64String(mySalt));
  48. Console.WriteLine();
  49. Console.WriteLine("Initilization Vector ({0} bytes): ", iv.Length);
  50. Console.WriteLine(Convert.ToBase64String(iv));
  51. Console.WriteLine();
  53. // Derive the passkey from a hash of the password plus salt with the number of hashing rounds.
  54. var deriveBytes = new Rfc2898DeriveBytes(myPassword, mySalt, 10000);
  56. // This gives us a derived byte key from our password.
  57. var aes256Key = deriveBytes.GetBytes(32);
  59. Console.WriteLine("Derived Key ({0} bytes): ", aes256Key.Length);
  60. Console.WriteLine(Convert.ToBase64String(aes256Key));
  61. Console.WriteLine();
  63. byte[] cipherTextBytes;
  65. using (var ms = new MemoryStream())
  66. {
  67. using (var cryptoStream = new CryptoStream(ms, transform.CreateEncryptor(aes256Key, iv), CryptoStreamMode.Write))
  68. {
  69. cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Count());
  70. // cryptoStream.FlushFinalBlock();
  71. }
  73. cipherTextBytes = ms.ToArray();
  75. Console.WriteLine("Encrypted Text ({0} bytes): ", cipherTextBytes.Length);
  76. Console.WriteLine(Convert.ToBase64String(cipherTextBytes));
  77. Console.WriteLine();
  78. }
  80. // this resets the algorithm. Normally, if we have a seperate encrypt / decrypt method
  81. // we would create a new instance of RijndaelManaged
  82. transform.Clear();
  84. using (var ms = new MemoryStream(cipherTextBytes))
  85. {
  86. using (var cryptoStream = new CryptoStream(ms, transform.CreateDecryptor(aes256Key, iv), CryptoStreamMode.Read))
  87. {
  88. var decryptedBytes = new byte[cipherTextBytes.Length];
  89. var length = cryptoStream.Read(decryptedBytes, 0, decryptedBytes.Length);
  91. var decryptedText = Encoding.UTF8.GetString(decryptedBytes.Take(length).ToArray());
  93. Console.WriteLine("Decrypted Text ({0} bytes): ", decryptedText.Length);
  94. Console.Write(decryptedText);
  95. Console.WriteLine();
  96. }
  97. }
  98. }
  99. }

Sample Output

Password (21 bytes):

Plain Text (355 bytes):
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi rutrum pulvinar purus, nec ornare neque cursus id. Nunc non tortor est. Morbi laoreet commodo tellus, et suscipit neque elementum eu. Sed velit lorem, ultricies id varius vitae, eleifend eget massa. Curabitur dignissim eleifend quam, sit amet interdum velit rutrum vel. Nulla nec enim tortor.

Salt (256 bytes):

Initilization Vector (16 bytes):

Derived Key (32 bytes):

Encrypted Text (368 bytes):

Decrypted Text (355 bytes):
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi rutrum pulvinar purus, nec ornare neque cursus id. Nunc non tortor est. Morbi laoreet commodo tellus, et suscipit neque elementum eu. Sed velit lorem, ultricies id varius vitae, eleifend eget massa. Curabitur dignissim eleifend quam, sit amet interdum velit rutrum vel. Nulla nec enim tortor.

That concludes my epic tour de AES in .NET. It’s like writing papers in college all over again. Hope somebody finds it useful.

EventHandler<T> or Action<T>

If you’ve used C# for any length of time, you’ve used events. Most likely, you wrote something like this:

  2. public class MyCoolCSharpClass {
  3. public event EventHandler MyCoolEvent;
  4. }
  6. public class MyOtherClass {
  7. public void MyOtherMethod(MyCoolCSharpClass obj)
  8. {
  9. obj.MyCoolEvent += WhenTheEventFires;
  10. }
  12. private void WhenTheEventFires(object sender, EventArgs args)
  13. {
  14. Console.WriteLine("Hello World!");
  15. }
  16. }

Later, you need parameters to be passed in along with the event, so you changed it to something like this:

  2. public event EventHandler<MyEventArgs> MyCoolEvent;
  4. public class MyEventArgs : EventArgs
  5. {
  6. public string Name { get; set; }
  7. public DateTime WhenSomethingHappened { get; set; }
  8. }
  9. ...
  10. private void WhenTheEventFires(object sender, MyEventArgs args)
  11. {
  12. var theCoolCSharpSendingClass = (MyCoolCSharpClass)sender;
  13. Console.WriteLine("Hello World! Good to meet you " + args.Name);
  14. }

You add two or three more events, some property change and changing events, and finally a class with about 4 properties, 3 events, and a little bit of code now has 3 supporting EventArgs classes, casts for every time you need the sender class instance (In this example, I’m assuming the event is always fired by MyCoolCSharpClass, and not through a method from a 3rd class). There’s a lot of code there to maintain even for just a simple class with some very simple functionality.

Lets look at this for a minute. First, EventHandler and EventHandler<T> are simply delegates, nothing more nothing less (If you’re not sure what a delegate is, don’t sweat it, it’s not really the point of this discussion). What makes the magic happen for events is that little event keyword the prefaces the event that turns that internally turns the delegate type into a subscribe-able field. Essentially, it simplifies adding and removing multiple methods that are all called when the event is invoked. With the introduction of generics in C# 2.0, and the introduction of LINQ in 3.5, we have generic forms of most of the delegates we could ever use in the form of Action<T1, T2, T3...> and Func<TRes, T1, T2...>. What this means, is that we can change an event declarations to use whatever delegate we want. Something like this is perfectly valid:

  2. public event Action<MyCoolCSHarpClass, string, DateTime> MyCoolEvent;

And what about when we subscribe? Well, now we get typed parameters:

  2. ...
  3. private void WhenTheEventFires(MyCoolCSHarpClass sender, string name, DateTime theDate)
  4. {
  5. Console.WriteLine("Hello World! Good to meet you " + name);
  6. }

That’s cool. I’ve now reduced the amount of code I have to maintain from 4 classes to 1 and I don’t have to cast my sender. As a matter of fact, I don’t even have to pass a sender. How often have you written an event that’s something like this:

  2. public event EventHandler TheTableWasUpdatedGoCheckIt;

Whoever is subscribed to this event doesn’t care about who sent it, or what data specifically was updated, all the subscribe cares about was that it was fired, nothing more than that. Even then, in a “you can only use EventHandler delegate world” you’re still stuck creating a method to subscribe to the event that looks like this:

  2. private void WhenTheTableWasUpdated(object sender, EventArgs args)
  3. {
  4. // Go check the database and update stuff...
  5. }

If we use what we’ve learned and change the event to something like this:

  2. public event Action TheTableWasUpdatedGoCheckIt;

We can write our method like this:

  2. private void WhenTheTableWasUpdated()
  3. {
  4. // Go check the database and update stuff...
  5. }

Since we never cared about the parameters in the first place.

Thats awesome fine and dandy, but just blindly replacing every instance of EventHandler delegates to Actions isn’t always the best idea, there are a few caveats:

First, there are some practical physical limitations of using Action<T1, T2, T2... > vs using a derived class of EventArgs, three main ones that I can think of:

  • If you change the number or types of parameters, every method that subscribes to that event will have to be changed to conform to the new signature. If this is a public facing event that 3rd party assemblies will be using, and there is any possibility that the number or type of arguments would change, its a very good reason to use a custom class that can later be inherited from to provide more parameters. Remember, you can still use an Action<MyCustomClass>, but deriving from EventArgs is still the Way Things Are Done
  • Using Action<T1, T2, T2... > will prevent you from passing feedback BACK to the calling method unless you have a some kind of object (with a Handled property for instance) that is passed along with the Action, and if you’re going to make a class with a handled property, making it derive from EventArgs is completely reasonable.
  • You don’t get named parameters by using Action<T1, T2 etc...> so if you’re passing 3 bool‘s, an int, two string‘s, and a DateTime, you won’t immediately know what the meaning of those values. Passing a custom args class provides meaning to those parameters.

Secondly, consistency implications. If you have a large system you’re already working with, it’s nearly always better to follow the way the rest of the system is designed unless you have an very good reason not too. If you have publicly facing events that need to be maintained, the ability to substitute derived classes for args might be important.

Finally, real life practice, I personally find that I tend to create a lot of one off events for things like property changes that I need to interact with (Particularly when doing MVVM with view models that interact with each other) or where the event has a single parameter. Most of the time these events take on the form of public event Action<[classtype], bool> [PropertyName]Changed; or public event Action SomethingHappened;. In these cases, there are two benefits that you might be able to guess from what you’ve already seen.

  • I get a type for the issuing class. If MyClass declares and is the only class firing the event, I get an explicit instance of MyClass to work with in the event handler.
  • For simple events such as property change events, the meaning of the parameters is obvious and stated in the name of the event handler and I don’t have to create a myriad of classes for these kinds of events.

Food for thought. If you have any comments, feel free to leave them in the comment section below.

Building a REALLY simple WCF P2P application

Often times when I’m looking at playing with a new technology that it becomes extremely difficult to find a simple stripped down easy-to-use chunk of code that also walks through, in-depth, the concepts and reasoning behind the code. This particular code chunk began as I started exploring the Microsoft WCF’s P2P networking services and found a rather distinct lack of code that explained the hows and whys of building a Peer to Peer network.

P2P Basics

Ok. Low level building blocks.

First, everybody (who’s interested in P2P) should already understand the client server paradigm, if not, there are many wonderfully detailed articles to explain the details. In short we have N clients all connected to 1 server. Client A sends a message to the server, the server decides what to do, and then may or may not send a response to Client A and / or Client B, Client C… etc… In a peer to peer network on the other hand, everyone is the client, everyone is a server. Because of this, the client / server naming scheme goes away by popular vote and everyone is called a node or a peer. When a node is connected to other nodes that whole group is called a mesh (or graph, or cloud).

Now, in a pure P2P network there is no central authority to help or govern how nodes find each other to form a mesh or how meshes are initially created.  Every node is connected to some number of other nodes, which are then connected to more nodes and so on. When one node wishes to communicate with another node (or nodes) the message is first passed on to the nodes that the first node knows about. These nodes in turn pass along the message on to other nodes that they know about and so on until finally everybody has seen the message.

One of the best, and probably most used, examples of a peer to peer network is the chat room. Until someone creates it, the chat room does not exist, but once it’s created people can be invited, join, and send messages that appear on everybody else’s screen. Even if the original person leaves the chat room, the room still exists as long as there are participants logged in. Once the last person leaves the chat room no longer exists.

Ping – The P2P Application

Note: I really dislike configuration files in demos or tutorials unless it’s a large application or showing how a particular aspect of a configuration file works. The fact that I can tell a static factory to build me something with a string name that corresponds to another file that somehow gets found, loaded, and happens to reference compiled type that then has a hidden class generated that implements that interface and is passed, bugs me when I’m trying to learn something.

If your going to follow along with this, your going to need .NET 3.5 installed and be running XP SP3, Vista, Win7 or Server 08. Visual Studio doesn’t hurt either.

To set up your project crack open Visual Studio and spin up a new Console Application, call it SimpleP2PExample. Once you have that open go over to the project, right click and Add Service Reference to System.ServiceModel, this allows you to use .NET’s WCF stuff in your app. You can choose to split up each class or interface into its own file or not: Up to you.

  1. //Contract for our network. It says we can 'ping'
  2. [ServiceContract(CallbackContract = typeof(IPing))]
  3. public interface IPing
  4. {
  5. [OperationContract(IsOneWay = true)]
  6. void Ping(string sender, string message);
  7. }

Alright, first of all, attributes. If you don’t know what they are, then here’s the low down mouthful one line explanation:

Attributes are essentially binary metadata associated with a class, method, property or whatever that provides additional information about whatever it’s “Decorating”.

The first attribute is the service contract. Wait a second. Contracts.

In our node-talking-to-other-nodes scenario, somehow they have to know how to talk to each other, if I asked what the size of the door was and you handed me a window, I have NO idea what that means or what that represents. A contract defines exactly what I’m telling you, what I’m expecting back, how, and when.

In this case we’re defining a contract that has one operation, a method called Ping. We know that when node A talks to node B that if node A says “Hey, Ping(“MyName”, “Hello.”) to Node B that node B will know what to do with that and how to pass it along to other nodes. It’s also specifies that I don’t expect Node B to give me anything back.

Now, the implementation.

  1. //implementation of our ping class
  2. public class PingImplementation : IPing
  3. {
  4. public void Ping(string sender, string message)
  5. {
  6. Console.WriteLine("{0} says: {1}", sender, message);
  7. }
  8. }

Fairly simple, whenever we receive a ping from another node, this method will be executed.

The Peer Class

Alright, now the fun, magic, and games begin. We’re going to create a class called peer, which will contain all our service start / stop code and also hold our implementation of PingImplementation.

  1. public class Peer
  2. {
  3. public string Id { get; private set; }
  5. public IPing Channel;
  6. public IPing Host;
  8. public Peer(string id)
  9. {
  10. Id = id;
  11. }
  12. }

In order to identify an individual node in the network, so that we know who’s who and don’t get everything mixed up, it’s customary to have a unique Id that’s assigned to the peer. Now, we have two IPing variables, the best way to describe them would be incoming and outgoing. An instance of the PingImplementation class will go in Host since it will be receiving any incoming communication from other nodes; The Channel is used to communicate out to other nodes and is built up via a factory.

  1. public void StartService()
  2. {
  3. var binding = new NetPeerTcpBinding();
  4. binding.Security.Mode = SecurityMode.None;
  6. var endpoint = new ServiceEndpoint(
  7. ContractDescription.GetContract(typeof(IPing)),
  8. binding,
  9. new EndpointAddress("net.p2p://SimpleP2P"));
  11. Host = new PingImplementation();
  13. _factory = new DuplexChannelFactory(
  14. new InstanceContext(Host),
  15. endpoint);
  17. var channel = _factory.CreateChannel();
  19. ((ICommunicationObject)channel).Open();
  21. // wait until after the channel is open to allow access.
  22. Channel = channel;
  23. }
  24. private DuplexChannelFactory<IPing>; _factory;

This is what we will use to start the peer.  This is the part where I’ve built up what could have been done in the configuration file with code instead.

Lets take it from the top.  First off, we have our binding; this defines what communication protocol we are going to be using, and as this is a PeerToPeer app… we use NetPeerTcpBinding().  You’ll also notice that in the next line I set the security mode to none; this is done for simplicities sake. There’s three types of security modes, including None, Message, Transport and TransportWithMessageCredential. Its a bit beyond the scope of this post, but Message security ensures that the message was not tampered with as it was passed from peer to peer, Transport security ensures that the connection between nodes is secure, and TransportWithMessageCredential does both.

Now, our application needs an endpoint, essentially, an service endpoint is a set of information that is exposed outside of the application (in this case on the network as well) so that others can access it. The endpoint defines the address it can be reached at, the contract, and the method that should be used to communicate. In this case, we build up our endpoint by using ContractDescription.GetContract to generate a contract class off of our IPing interface, our network type binding, and an endpoint address where this endpoint can be reached.

Finally, we create a new instance of our PingImplementation class as the Host, and we create our channel factory. A DuplexChannelFactory allows for two way communication, the first parameter is the object that you want to receive incoming calls, and the endpoint is where those calls are coming from. The factory then creates the channel and a whole bunch of magical things happen.

If you’ll remember, our channel is of type IPing, the Duplex Factory performs some magic and generates a concrete implementation of your interface (it also implements ICommunicationObject, which is why you’ll sometimes see people create another interface called something like “IPingChannel : IPing, ICommunicationObject”, it does make it so that you don’t have to cast it, but for the purpose of this post it’s not necessary). Imagine that it takes your interface, implements all the methods and properties with all the cool DuplexChannel code needed to talk back and fourth, creates an instance, and returns it to you.

Finally, I call open on my channel to let the world see that my brand new channel and endpoint are ready for business.

  1. public void StopService()
  2. {
  3. ((ICommunicationObject)Channel).Close();
  4. if (_factory != null)
  5. _factory.Close();
  6. }

Now, it’s all well and good, until your done. Then you need to close you channel and factory, this should be pretty self explanatory at this point.  Remember our channel is an ICommunicationObject in addition to being a IPing object, so we cast and close, then check to see if our factory is null, and if not, close that as well.

Threading The Peer Class

Something I chose to do was make the peer threaded. This allows me to drop it into an application, in a thread, and receive and push stuff into it at my leisure. To do this I add in a:

  1. private readonly AutoResetEvent _stopFlag = new AutoResetEvent(false);

This will allow me to block a method of the thread until I decide fire it (When I stop the peer).

  1. public void Run()
  2. {
  3. Console.WriteLine("[ Starting Service ]");
  4. StartService();
  6. Console.WriteLine("[ Service Started ]");
  7. _stopFlag.WaitOne();
  9. Console.WriteLine("[ Stopping Service ]");
  10. StopService();
  12. Console.WriteLine("[ Service Stopped ]");
  13. }
  15. public void Stop()
  16. {
  17. _stopFlag.Set();
  18. }

The run method embodies the lifecycle of this peer. When the peer thread starts into the run method it will start the service, wait until the stop flag is fired (when the Stop() method is called), and then stop and dispose the service.

Putting It All Together
  1. class Program
  2. {
  3. static void Main(string[] args)
  4. {
  5. if (Process.GetProcessesByName(Process.GetCurrentProcess().ProcessName).Count() <= 1)
  6. {
  7. for (int i = 0; i < 4; i++)
  8. {
  9. Process.Start("SimpleP2PExample.exe");
  10. }
  11. }
  13. new Program().Run();
  14. }
  16. public void Run()
  17. {
  18. Console.WriteLine("Starting the simple P2P demo.");
  20. var peer = new Peer("Peer(" + Guid.NewGuid() + ")");
  21. var peerThread = new Thread(peer.Run) {IsBackground = true};
  22. peerThread.Start();
  24. //wait for the server to start up.
  25. Thread.Sleep(1000);
  27. while (true)
  28. {
  29. Console.Write("Enter Something: ");
  30. string tmp = Console.ReadLine();
  32. if (tmp == "") break;
  34. peer.Channel.Ping(peer.Id, tmp);
  35. }
  37. peer.Stop();
  38. peerThread.Join();
  39. }
  40. }

From top to bottom, the Main method of the application checks to see if it’s the first one of this application to have started up, if it its, then it starts four additional processes (Note that if you called your project something other than SimpleP2PExample, you will need to replace the the string in Process.Start to be the name as the executable file your project generates).

The run method is also fairly simple, it creates a new instance of the Peer class, assigns it a GUID, creates a thread for the peer’s run method, starts up the thread and pauses to wait for the peer’s thread to start.  We then enter a loop until the user presses enter without inputting any text. Any text that is put in is transmitted over the peer channel using the peer’s id and the message. Once we’ve finished, we stop the session and wait for the peerThread to exit and join back up with the main thread. We then exit.

Wrap Up

I hope this helps someone out there get a better understanding of the basic concepts of a simple Peer to Peer application. Feel free to leave feedback!

Paul Rohde