On Wed, 2013-08-14 at 09:31 +0300, Artem Bityutskiy wrote:
But this all is not the point, these are additional goodies. The
main
point of bmaptool is in having the bmap file, and _then_ you get real
speed-up, based on the sparseness of the original image.
Just as a rough demonstration, as I described in the original e-mail,
I've "reconstructed" the original image sparseness using 'cp
--sparse=always', and then created the bmap file, which at the end of
this e-mail.
You can see that only 33.6% of the image contains non-zero blocks, so
with bmap file you'd copy only 688.6MiB instead of 2.1GiB. This is where
the speed would come from. This is why I got a seemingly unrealistically
fast flashing speed in the original e-mail.
But sure, 33.6% is incorrect number, the real number will probably be
slightly higher. The explanations why can be found here:
https://source.tizen.org/documentation/reference/bmaptool/introduction
in the "Reconstructing sparse files" section.
It is technically easy to add bmap file generation to the fedora image
build tools and generate the bmap file along with the image. I could try
to do this if people are interested.
Here is the bmap file I got:
<?xml version="1.0" ?>
<!-- This file contains the block map for an image file, which is basically
a list of useful (mapped) block numbers in the image file. In other words,
it lists only those blocks which contain data (boot sector, partition
table, file-system metadata, files, directories, extents, etc). These
blocks have to be copied to the target device. The other blocks do not
contain any useful data and do not have to be copied to the target
device.
The block map an optimization which allows to copy or flash the image to
the image quicker than copying of flashing the entire image. This is
because with bmap less data is copied: <MappedBlocksCount> blocks instead
of <BlocksCount> blocks.
Besides the machine-readable data, this file contains useful commentaries
which contain human-readable information like image size, percentage of
mapped data, etc.
The 'version' attribute is the block map file format version in the
'major.minor' format. The version major number is increased whenever an
incompatible block map format change is made. The minor number changes
in case of minor backward-compatible changes. -->
<bmap version="1.3">
<!-- Image size in bytes: 2.0 GiB -->
<ImageSize> 2147483648 </ImageSize>
<!-- Size of a block in bytes -->
<BlockSize> 4096 </BlockSize>
<!-- Count of blocks in the image file -->
<BlocksCount> 524288 </BlocksCount>
<!-- Count of mapped blocks: 688.6 MiB or 33.6% -->
<MappedBlocksCount> 176288 </MappedBlocksCount>
<!-- The checksum of this bmap file. When it is calculated, the value of
the SHA1 checksum has be zero (40 ASCII "0" symbols). -->
<BmapFileSHA1> 75b48dc596a5e92d7cc4935d8fcc3a91c2e48b0f </BmapFileSHA1>
<!-- The block map which consists of elements which may either be a
range of blocks or a single block. The 'sha1' attribute (if present)
is the SHA1 checksum of this blocks range. -->
<BlockMap>
<Range sha1="ce5cc4f31d623a34c01f791e6e5c1ee65456044b"> 0-15
</Range>
<Range sha1="4ca6ee9c01354785502ba201a7c03cdd4ffbaedb"> 240-1967
</Range>
<Range sha1="6f0869f38618931042495803f5b4c02f2d7d03e9"> 8224-9583
</Range>
<Range sha1="b7650fa6a6bb703e30438ae22ca7a53c107f72ad"> 9632-11599
</Range>
<Range sha1="4e04cad0efb9194e7e00d407caaf89c9bc19042d">
11632-11775 </Range>
<Range sha1="feebd3924ce0db12f1d7186577ca54985984ab69">
33008-33023 </Range>
<Range sha1="ce74e311d9c1316f03b3b5a64225b607196419b9">
33136-33567 </Range>
<Range sha1="1e6a1f19428243b09da296e5e5494d9443b90c64">
34032-34095 </Range>
<Range sha1="9b2b5218b79d1b51e00d67814a8eea41d5cc430f">
34544-34591 </Range>
<Range sha1="f36da167d811feba274928d5c293b569b9f8c0aa">
34656-34671 </Range>
<Range sha1="2ffd3f90346ba2d0a9841865ccca0adfa19b830e">
35056-35087 </Range>
<Range sha1="48a82266b1351a543a169df69274c7f82c743988">
35568-35615 </Range>
<Range sha1="71ffc02cadefd321af333fa4b8e54ca6a2a48fbe">
35888-35903 </Range>
<Range sha1="e3b51cd7304df8df536e9ebd6d84d6d0d516d0a0">
36080-36255 </Range>
<Range sha1="6939de8eed4c6c35691f6b6ccf805e7d7a141b70">
36304-36511 </Range>
<Range sha1="9d2b2e2c0cc771f2a2684317ffdaa8557d5962e1">
36592-37071 </Range>
<Range sha1="5ccf6bca3a0140f3f549b67cf114e8847d8fd82e">
37104-55551 </Range>
<Range sha1="a3259735e513eb38f27058e6c87bd555d060615a">
62656-62687 </Range>
<Range sha1="27c3ac0865d0b12180395e2d083f7bbdc0440bb3">
62992-63023 </Range>
<Range sha1="bfe6dbe0c325a0a95a6cf6d62226287f4dd0b99d">
64144-64175 </Range>
<Range sha1="ac97574508a8bd42f39bed6ccbe0ab1e57cbfd54">
65120-98559 </Range>
<Range sha1="2351cae4967fe643d5003b6f526c811cd311df45">
98672-100975 </Range>
<Range sha1="6a47e91dc5be78d69761ab762fc2e3ae1ea9bc8e">
101040-101615 </Range>
<Range sha1="4e9bbbb0f3b487efd37cc54808a371d27eac4c78">
101808-123759 </Range>
<Range sha1="05a6b82bcd9672a04252bc090b368e6deb3084b1">
123952-124015 </Range>
<Range sha1="531f3dcb1cefdc515233da007069761849058b13">
124064-141151 </Range>
<Range sha1="626d4a7ed0e01815230b1fc95e22e7095b3c2c5a">
141392-143327 </Range>
<Range sha1="04c93f6e63aec7320088037fab3b5d0edb994d23">
143360-146319 </Range>
<Range sha1="949d1861d7dde9c2ad916a0807ac1b2db6ee2002">
146624-155391 </Range>
<Range sha1="4e5395a152a382e5e487e24af9d2ed8930eb4531">
155888-164095 </Range>
<Range sha1="484548b0536d1276575f620139ff3596a455b017">
164208-165983 </Range>
<Range sha1="e44b975f3d627e4f8df3442d27ca0aeaa7329af5">
166016-166047 </Range>
<Range sha1="ce95c6eb68528b4d310d0dc087ba595b43473fef">
166080-167103 </Range>
<Range sha1="d42280d545afdc3c5fb6ae5a0e6e7e3430cb4d23">
167568-196143 </Range>
<Range sha1="2856f3fac20726e5c647d32fe4757f1b091309b0">
196848-213247 </Range>
<Range sha1="10614aa162d95797e64dff049707ae788a19d40f">
221424-223311 </Range>
<Range sha1="a4c0fef5c6c9a013fc641016face92e92329864b">
223344-223391 </Range>
<Range sha1="b7375c8568a14fb1217a1e31f52cfdffbdd9ce30">
223472-227375 </Range>
<Range sha1="855e049fb6ace103d4569465b0b06df47628b76f">
229616-229631 </Range>
<Range sha1="0c900792e5cc9613496ab53d1f30bb94b1c9ad9d">
295152-295167 </Range>
<Range sha1="2efc62831e5d49db8550c7ef401bc1c1c2a27a78">
295280-295295 </Range>
</BlockMap>
</bmap>
--
Best Regards,
Artem Bityutskiy