hunk ./Migrate.lhs 18
-  <p>HAppS applications, like any application with persistent data storage, are faced with the issue of migrating existing data when the format of the persistent data is changed. This tutorial will explore the automatic binary serialization and migration facilities provided by HAppS-Data.</p>
-
+  <p>HAppS applications, like any application with persistent data storage, are faced with the issue of migrating existing data when the format of the persistent data is changed. This tutorial will explore the binary serialization and migration facilities provided by HAppS-Data. If you think I am doing it all wrong, please let me know. Writing this tutorial is the extent of my experience using the <kbd>HApps-Data</kbd> migration facilities.</p>
+<!--
hunk ./Migrate.lhs 26
-<!--
+
hunk ./Migrate.lhs 31
+<h3>Requirements</h3>
+
+<p>This tutorial only uses the <kbd>HAppS-Data</kbd> (and dependencies) portion of <kbd>HAppS</kbd>. It has been tested with <kbd>HAppS-Data 0.9.3</kbd>. The first three lines of the module look like this:</p>
hunk ./Migrate.lhs 41
-<p>The most straight forward way to serialize data is to use the familiar Read and Show classes. Simply use <code>show</code> to turn a value into a <code>String</code>, and <code>read</code> to turn a <code>String</code> back into a value. This method has a few issues:</p>
+<p>The most obvious way to serialize data in Haskell is to use the familiar <code>Read</code> and <code>Show</code> classes. Simply use <code>show</code> to turn a value into a <code>String</code>, and <code>read</code> to turn a <code>String</code> back into a value. This method has three serious flaws however:</p>
hunk ./Migrate.lhs 43
-<ul>
- <li>The law <code>read . show == id</code> does hold for all Show/Read instances
- <li>Poor space usage
- <li>No migration path when types change
-</ul>
+<ol>
+ <li>The law <code>read . show == id</code> does not hold for all Show/Read instances. 
+ <li>The serialized representation is rather verbose
+ <li>No migration path when types change, leaving your old data inaccessible
+</ol>
hunk ./Migrate.lhs 49
-<p><kbd>HAppS-Data</kbd> provides a <code>Serialize</code> class which addresses those three issues. From an end user point of view there are three items of interest:</p>
+<p><kbd>HAppS-Data</kbd> provides a <code>Serialize</code> class which addresses these three issues. From an end user point of view the <code>Serialize</code> functionality provides three items of interest:</p>
hunk ./Migrate.lhs 53
-  <li>the <code>deriveSerialize</code> function
+  <li>The <code>deriveSerialize</code> function
hunk ./Migrate.lhs 57
-> class (Typeable a, Version a) => Serialize a
+> class (Typeable a, Version a) => Serialize a where 
+>   ...
hunk ./Migrate.lhs 67
-<p>The <code>Version</code> superclass is used during data migration. The <code>serialize</code> and <code>deserialize</code> functions are the counterparts to <code>show</code> and <code>read</code>.</p>
+<p>The <code>Version</code> superclass is used during data migration. The <code>serialize</code> and <code>deserialize</code> functions are the counterparts to <code>show</code> and <code>read</code>. <code>deriveSerialize</code> is a Template Haskell function which provides functionality similar to <code>deriving (Read, Show)</code>.</p>
hunk ./Migrate.lhs 74
+>
hunk ./Migrate.lhs 79
-> newtype VersionId a = VersionId {unVersion :: Int} deriving (Num,Read,Show,Eq)
->
hunk ./Migrate.lhs 82
+> newtype VersionId a = VersionId {unVersion :: Int} deriving (Num,Read,Show,Eq)
hunk ./Migrate.lhs 87
-  <li>primitives whose layout will never change
+  <li>primitives whose layout will never change, and, hence, will never need to be migrated
hunk ./Migrate.lhs 91
-<p>The <code>Versioned</code> constructor takes two arguments. The first argument is a version number which you increment when you make an change to the data-type. The second argument is an indicator of the previous version of the data-type. This indicator will be used during the migration process, which will be explained later.</p>
+<p>The <code>Versioned</code> constructor takes two arguments. The first argument is a version number which you increment when you make an change to the data-type. The second argument is an indicator of the previous version of the data-type. The exact details are covered in the next section.
hunk ./Migrate.lhs 108
-<p>The <code>deriveAll</code> template haskell function is similar to the normal haskell deriving clause, except it also has the ability to derive <code>Default</code> instances, and it always derives <code>Typeable</code> and <code>Data</code> instances.</p>
+<p>The <code>deriveAll</code> template haskell function is similar to the normal haskell deriving clause, except it also has the ability to derive <code>Default</code> instances. Additionally, it always derives <code>Typeable</code> and <code>Data</code> instances even though they are not explicitly listed.</p>
hunk ./Migrate.lhs 118
-<p>We indicate that <code>Beep</code> and <code>Foo</code> are non-primitive data-types, with <code>VersionId 0</code> and that there are no previous versions of these types.</p>
+<p>We want to indicate that <code>Beep</code> and <code>Foo</code> are non-primative types, so we use the <code>Versioned</code> constructor. Next we specify a version number for the type. It could be anything, but <code>0</code> is the most sensible choice. Since there are now previous versions of these types we mark the previous type as <code>Nothing</code>.
hunk ./Migrate.lhs 120
-<p>This is actually the default definition for mode if none is supplied:</p>
+<p>For all non-primitive types the initial version of <code>Versioned 0 Nothing</code> is sensible. So the <code>Version</code> class provides it as a default value for <code>mode</code>:</p>
hunk ./Migrate.lhs 128
-<p>So we could shorten things to:</p>
+<p>Hence, we could shorten our <code>Version</code> instances from above to:</p>
hunk ./Migrate.lhs 148
-<p>We see that <code>Beep</code> serializes to 9 bytes. The first 8 bytes represent the <code>VersionId</code>. <code>VersionId</code> is basically an <code>Int</code>, and <code>Int</code>s are treated as a 64-bit values to avoid cross-platform issues. The final byte indicates which constructor of <code>Beep</code> was used. In this case the zeroth constructor was used.</p>
+<p>We see that <code>Beep</code> serializes to 9 bytes. The first 8 bytes represent the <code>VersionId</code>. <code>VersionId</code> is basically an <code>Int</code>, and the serialization code always treats <code>Int</code>s as a 64-bit values to avoid cross-platform issues. The final byte indicates which constructor of <code>Beep</code> was used. In this case the zeroth constructor was used.</p>
hunk ./Migrate.lhs 150
-<p>At first it may seem like we don't have enough information here to deserialize the data, after all there are no type indicators, etc. But this situation is no different than doing <code>read "1" :: Int</code>. Because we know the type of the value we want to be reading it at compile time, we do not need to record that information in the stored data. Hence we can just do:</p>
+<p>At first it may seem like we don't have enough information here to deserialize the data, after all there are no type names, constructors, etc. But deserializing these bytes is no different than doing <code>read "1" :: Int</code>. Because we know the type of the value we want to be reading at compile time, we do not need to record that information in the stored data. We just do:</p>
hunk ./Migrate.lhs 158
-<p>As a side note, it is interesting to note that Strings are stored in the persistent storage very efficiently. In fact, they are stored as efficiently as <code>ByteString</code>s because they are first converted to a <code>ByteString</code>. </p>
+<p>As a side note, <code>String</code>s are serialized to a very compact representation. In fact, they are stored as compactly as <code>ByteString</code>s because they are first converted to a <code>ByteString</code>. </p>
hunk ./Migrate.lhs 170
-<p>However, you will still save space in memory by using <code>ByteString</code> instead of <code>String</code>.
+<p>So, if you application is best served by using <code>String</code>s instead of <code>ByteString</code>s, you do not have to take an extra steps to ensure that the serialized data is compactly represented.</p>
hunk ./Migrate.lhs 174
-<p>Let's add another constructor to the <code>Beep</code> type. In this example, we will actually create a whole new type named <code>Beep'</code>, which is similar to the old type, but has an additional constructor <code>BeepBeep</code>.</p>
+<p>Let's say we want to add another constructor to the <code>Beep</code> type. As a first pass, we will actually create a whole new type named <code>Beep'</code>, which is similar to the old type, but has an additional constructor <code>BeepBeep</code>.</p>
hunk ./Migrate.lhs 197
-<p>The <code>Migrate</code> class is quite simple, it contains a single function, <code>migrate</code> which migrates something of type <code>a</code> to type <code>b</code>. In our current example, this is simply:</p>
+<p>The <code>Migrate</code> class is quite simple, it contains a single function, <code>migrate</code> which migrates something of type <code>a</code> to type <code>b</code>. In our current example, all we need is:</p>
hunk ./Migrate.lhs 202
-<p>Here is a simple example where we serialize a value of type <code>Beep</code> and deserialize it as type <code>Beep'</code>. The migration happens automatically in the <code>deserialize</code> function.</p>
+<p>We can demonstrate migration by serializing a value of type <code>Beep</code> and deserializing it as type <code>Beep'</code>. The migration happens automatically in the <code>deserialize</code> function.</p>
hunk ./Migrate.lhs 207
-*Main> 
+*Main>
hunk ./Migrate.lhs 210
-<p>When <code>deserialize</code> tries to deserialize the data produced by <code>serialize Beep</code>, it will first check the version number. When it sees that the version number in the stored data is lower than the version number of the current type it will instead try to decode it as the type you specified as the previous version. If the version associated with the previous type is still higher than the value in the data, the migration code will recurse until it finds a matching version number. Once it finds a matching version number, it will call the corresponding deserialization "instance" to decode the old data. Then as the recursion unwinds, it will apply the <code>migrate</code> function to migrate the data to newer and newer formats until it reaches the newest format. </p>
+<p>When <code>deserialize</code> tries to deserialize the data produced by <code>serialize Beep</code>, it will first check the version number. When it sees that the version number in the stored data is lower than the version number of the current type it will instead try to decode it as the type you specified as the previous version. If the version associated with the previous type is still higher than the value in the serialized data, the migration code will recurse until it finds a matching version number. Once it finds a matching version number, it will call the corresponding deserialization "instance" to decode the old data. Then as the recursion unwinds, it will apply the <code>migrate</code> function to migrate the data to newer and newer formats until it reaches the newest format.</p>
hunk ./Migrate.lhs 214
-<p>The big issue in the above example is that we changed the name of the type and its constructors. That is not very convenient in a real application where you have a multitude of references to the old name. Fortunately, we do not actually have to change the name of the type. As we saw in the beginning, the name of the type and the names of the constructors are not actually stored in the serialized data. So, instead we can change the name of the old type from <code>Beep</code> to <code>OldBeep</code> and update its constructor as well.</p>
+<p>A big issue in the above example is that when we added the new constructor we also changed the name of the type and its existing constructors. That is not very convenient in a real application where you have a multitude of references to the old names.</p>
+
+<p>Fortunately, we do not have to change the name of the type to add a new constructor. As we saw in the beginning, the name of the type and the names of the constructors are not actually stored in the serialized data. So, instead we can change the name of the old type from <code>Beep</code> to <code>OldBeep</code> and update its constructor as well.</p>
hunk ./Migrate.lhs 226
-<p>Because <code>OldBeep</code> and <code>Beep</code> have the same shape, we can do this:</p>
+<p>Because <code>OldBeep</code> and <code>Beep</code> have the same shape, they will serialize to the same byte sequence:
+
+<pre>
+*Main> Data.ByteString.Lazy.unpack $ serialize OldBeep
+[0,0,0,0,0,0,0,0,0]
+*Main> Data.ByteString.Lazy.unpack $ serialize Beep
+[0,0,0,0,0,0,0,0,0]
+*Main> 
+</pre>
+
+<p>that means we can serialize an <code>OldBeep</code> value and then deserialize it as a <code>Beep</code> value, like this:</p>
hunk ./Migrate.lhs 244
+<p>Note that this is not the same as migration. Here we are just exploiting the fact that because the type name and constructor names are not encoded in the serialized data we can change those names and still be able to deserialize the data.</p>
+
+<h4>Full Migration Example #1</h4>
+
hunk ./Migrate.lhs 282
-<p>Keeping all the revisions of your type in one file, and changing name is quite tedious and hard to manage after a while. So instead, lets put all of our types in a separate file (or files), such as <kbd>Types.lhs</kbd>.</p>
+<p>Keeping all the revisions of your type in one file, and changing the name of the type and its constructors every revision is tedious and hard to manage. Instead, we can use a system where we rename the files that contain our types. To start, we will put the types we want to serialize in a separate file (or files), such as <kbd>Types.lhs</kbd>.</p>
hunk ./Migrate.lhs 303
-<p>It is also possible to serialize datatypes from 3rd party libraries, provided those types have <code>Data</code> and <code>Typeable</code> instances. There is a caveat with this however. If the third party library changes the type, then you unexpectedly will not be able to read your data. This is not a fatal flaw however. You can simply copy the old type into a local module, and then use that to migrate to the new upstream version.</p>
-
-<p>However, you may consider a policy of only using locally defined datatypes.</p>
+<p>It is also possible to serialize datatypes from 3rd party libraries, provided those types have <code>Data</code> and <code>Typeable</code> instances. There is a caveat with this however. If the third party library changes the type, then you will not be able to read your data. This is not a fatal flaw however. You can simply copy the old type definition into a local module, and then migrate the old data to the new format.</p>
hunk ./Migrate.lhs 308
- <li>Put the types you will serialize in one or more separate files
- <li>Deploy
- <li>Copy current type files to sequential versions and create new current type files which re-export all the types. You can skip this step if the current type file only contains re-exports. I.e., if no type changes were made to that type file during that iteration.
+ <li>Put the types you will serialize in one or more files which only contain types
+ <li>Deploy your web 2.718 killer app
+ <li>Before you do any more development, copy the current type files to sequential versions and create new current type files which re-export all the types. You can skip this step if the current type file only contains re-exports. i.e., if no type changes were made to that type file during the previous iteration.
hunk ./step2/Types.lhs 9
-<p>We import qualified the old <kbd>Types_000</kbd> as T0 to avoid name clashes.</p>
+<p>We import the old <kbd>Types_000</kbd> qualified as <code>T0</code> to avoid name clashes.</p>
hunk ./step2/Types.lhs 13
-<p>Since we are only modifying Foo, we can import and re-export the old version of Beep unmodified.</p>
+<p>Since we are only modifying Foo, we can import and re-export the old version of Beep unmodified (also see the module export list above):</p>
hunk ./step2/Types.lhs 17
-<p> Then we extend <code>Foo</code>:</p>
+<p> Then we extend <code>Foo</code> with the new constructor <code>Bop Int</code>:</p>
hunk ./step2/Types.lhs 27
-<p>The <code>Version</code> instance indicates that the previous version was T0.Foo.</p>
+<p>Next we create a <code>Version</code> instance which indicates that the previous version of <code>Foo</code> is <code>T0.Foo</code>.</p>
hunk ./step2/Types.lhs 36
-<p>And then we specify how to migrate the old data:</p>
+<p>And, finally, we specify how to migrate the old data:</p>