/usr/share/doc/bzip2/manual.html is in bzip2-doc 1.0.6-8.1.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 | <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>bzip2 and libbzip2, version 1.0.6</title>
<meta name="generator" content="DocBook XSL Stylesheets V1.75.2">
<style type="text/css" media="screen">/* Colours:
#74240f dark brown h1, h2, h3, h4
#336699 medium blue links
#339999 turquoise link hover colour
#202020 almost black general text
#761596 purple md5sum text
#626262 dark gray pre border
#eeeeee very light gray pre background
#f2f2f9 very light blue nav table background
#3366cc medium blue nav table border
*/
a, a:link, a:visited, a:active { color: #336699; }
a:hover { color: #339999; }
body { font: 80%/126% sans-serif; }
h1, h2, h3, h4 { color: #74240f; }
dt { color: #336699; font-weight: bold }
dd {
margin-left: 1.5em;
padding-bottom: 0.8em;
}
/* -- ruler -- */
div.hr_blue {
height: 3px;
background:#ffffff url("/images/hr_blue.png") repeat-x; }
div.hr_blue hr { display:none; }
/* release styles */
#release p { margin-top: 0.4em; }
#release .md5sum { color: #761596; }
/* ------ styles for docs|manuals|howto ------ */
/* -- lists -- */
ul {
margin: 0px 4px 16px 16px;
padding: 0px;
list-style: url("/images/li-blue.png");
}
ul li {
margin-bottom: 10px;
}
ul ul {
list-style-type: none;
list-style-image: none;
margin-left: 0px;
}
/* header / footer nav tables */
table.nav {
border: solid 1px #3366cc;
background: #f2f2f9;
background-color: #f2f2f9;
margin-bottom: 0.5em;
}
/* don't have underlined links in chunked nav menus */
table.nav a { text-decoration: none; }
table.nav a:hover { text-decoration: underline; }
table.nav td { font-size: 85%; }
code, tt, pre { font-size: 120%; }
code, tt { color: #761596; }
div.literallayout, pre.programlisting, pre.screen {
color: #000000;
padding: 0.5em;
background: #eeeeee;
border: 1px solid #626262;
background-color: #eeeeee;
margin: 4px 0px 4px 0px;
}
</style>
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div lang="en" class="book" title="bzip2 and libbzip2, version 1.0.6">
<div class="titlepage">
<div>
<div><h1 class="title">
<a name="userman"></a>bzip2 and libbzip2, version 1.0.6</h1></div>
<div><h2 class="subtitle">A program and library for data compression</h2></div>
<div><div class="authorgroup"><div class="author">
<h3 class="author">
<span class="firstname">Julian</span> <span class="surname">Seward</span>
</h3>
<div class="affiliation"><span class="orgname">http://www.bzip.org<br></span></div>
</div></div></div>
<div><p class="releaseinfo">Version 1.0.6 of 6 September 2010</p></div>
<div><p class="copyright">Copyright © 1996-2010 Julian Seward</p></div>
<div><div class="legalnotice" title="Legal Notice">
<a name="id537185"></a><p>This program, <code class="computeroutput">bzip2</code>, the
associated library <code class="computeroutput">libbzip2</code>, and
all documentation, are copyright © 1996-2010 Julian Seward.
All rights reserved.</p>
<p>Redistribution and use in source and binary forms, with
or without modification, are permitted provided that the
following conditions are met:</p>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc"><p>Redistributions of source code must retain the
above copyright notice, this list of conditions and the
following disclaimer.</p></li>
<li class="listitem" style="list-style-type: disc"><p>The origin of this software must not be
misrepresented; you must not claim that you wrote the original
software. If you use this software in a product, an
acknowledgment in the product documentation would be
appreciated but is not required.</p></li>
<li class="listitem" style="list-style-type: disc"><p>Altered source versions must be plainly marked
as such, and must not be misrepresented as being the original
software.</p></li>
<li class="listitem" style="list-style-type: disc"><p>The name of the author may not be used to
endorse or promote products derived from this software without
specific prior written permission.</p></li>
</ul></div>
<p>THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE.</p>
<p>PATENTS: To the best of my knowledge,
<code class="computeroutput">bzip2</code> and
<code class="computeroutput">libbzip2</code> do not use any patented
algorithms. However, I do not have the resources to carry
out a patent search. Therefore I cannot give any guarantee of
the above statement.
</p>
</div></div>
</div>
<hr>
</div>
<div class="toc">
<p><b>Table of Contents</b></p>
<dl>
<dt><span class="chapter"><a href="#intro">1. Introduction</a></span></dt>
<dt><span class="chapter"><a href="#using">2. How to use bzip2</a></span></dt>
<dd><dl>
<dt><span class="sect1"><a href="#name">2.1. NAME</a></span></dt>
<dt><span class="sect1"><a href="#synopsis">2.2. SYNOPSIS</a></span></dt>
<dt><span class="sect1"><a href="#description">2.3. DESCRIPTION</a></span></dt>
<dt><span class="sect1"><a href="#options">2.4. OPTIONS</a></span></dt>
<dt><span class="sect1"><a href="#memory-management">2.5. MEMORY MANAGEMENT</a></span></dt>
<dt><span class="sect1"><a href="#recovering">2.6. RECOVERING DATA FROM DAMAGED FILES</a></span></dt>
<dt><span class="sect1"><a href="#performance">2.7. PERFORMANCE NOTES</a></span></dt>
<dt><span class="sect1"><a href="#caveats">2.8. CAVEATS</a></span></dt>
<dt><span class="sect1"><a href="#author">2.9. AUTHOR</a></span></dt>
</dl></dd>
<dt><span class="chapter"><a href="#libprog">3.
Programming with <code class="computeroutput">libbzip2</code>
</a></span></dt>
<dd><dl>
<dt><span class="sect1"><a href="#top-level">3.1. Top-level structure</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#ll-summary">3.1.1. Low-level summary</a></span></dt>
<dt><span class="sect2"><a href="#hl-summary">3.1.2. High-level summary</a></span></dt>
<dt><span class="sect2"><a href="#util-fns-summary">3.1.3. Utility functions summary</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#err-handling">3.2. Error handling</a></span></dt>
<dt><span class="sect1"><a href="#low-level">3.3. Low-level interface</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#bzcompress-init">3.3.1. BZ2_bzCompressInit</a></span></dt>
<dt><span class="sect2"><a href="#bzCompress">3.3.2. BZ2_bzCompress</a></span></dt>
<dt><span class="sect2"><a href="#bzCompress-end">3.3.3. BZ2_bzCompressEnd</a></span></dt>
<dt><span class="sect2"><a href="#bzDecompress-init">3.3.4. BZ2_bzDecompressInit</a></span></dt>
<dt><span class="sect2"><a href="#bzDecompress">3.3.5. BZ2_bzDecompress</a></span></dt>
<dt><span class="sect2"><a href="#bzDecompress-end">3.3.6. BZ2_bzDecompressEnd</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#hl-interface">3.4. High-level interface</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#bzreadopen">3.4.1. BZ2_bzReadOpen</a></span></dt>
<dt><span class="sect2"><a href="#bzread">3.4.2. BZ2_bzRead</a></span></dt>
<dt><span class="sect2"><a href="#bzreadgetunused">3.4.3. BZ2_bzReadGetUnused</a></span></dt>
<dt><span class="sect2"><a href="#bzreadclose">3.4.4. BZ2_bzReadClose</a></span></dt>
<dt><span class="sect2"><a href="#bzwriteopen">3.4.5. BZ2_bzWriteOpen</a></span></dt>
<dt><span class="sect2"><a href="#bzwrite">3.4.6. BZ2_bzWrite</a></span></dt>
<dt><span class="sect2"><a href="#bzwriteclose">3.4.7. BZ2_bzWriteClose</a></span></dt>
<dt><span class="sect2"><a href="#embed">3.4.8. Handling embedded compressed data streams</a></span></dt>
<dt><span class="sect2"><a href="#std-rdwr">3.4.9. Standard file-reading/writing code</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#util-fns">3.5. Utility functions</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#bzbufftobuffcompress">3.5.1. BZ2_bzBuffToBuffCompress</a></span></dt>
<dt><span class="sect2"><a href="#bzbufftobuffdecompress">3.5.2. BZ2_bzBuffToBuffDecompress</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#zlib-compat">3.6. zlib compatibility functions</a></span></dt>
<dt><span class="sect1"><a href="#stdio-free">3.7. Using the library in a stdio-free environment</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#stdio-bye">3.7.1. Getting rid of stdio</a></span></dt>
<dt><span class="sect2"><a href="#critical-error">3.7.2. Critical error handling</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#win-dll">3.8. Making a Windows DLL</a></span></dt>
</dl></dd>
<dt><span class="chapter"><a href="#misc">4. Miscellanea</a></span></dt>
<dd><dl>
<dt><span class="sect1"><a href="#limits">4.1. Limitations of the compressed file format</a></span></dt>
<dt><span class="sect1"><a href="#port-issues">4.2. Portability issues</a></span></dt>
<dt><span class="sect1"><a href="#bugs">4.3. Reporting bugs</a></span></dt>
<dt><span class="sect1"><a href="#package">4.4. Did you get the right package?</a></span></dt>
<dt><span class="sect1"><a href="#reading">4.5. Further Reading</a></span></dt>
</dl></dd>
</dl>
</div>
<div class="chapter" title="1. Introduction">
<div class="titlepage"><div><div><h2 class="title">
<a name="intro"></a>1. Introduction</h2></div></div></div>
<p><code class="computeroutput">bzip2</code> compresses files
using the Burrows-Wheeler block-sorting text compression
algorithm, and Huffman coding. Compression is generally
considerably better than that achieved by more conventional
LZ77/LZ78-based compressors, and approaches the performance of
the PPM family of statistical compressors.</p>
<p><code class="computeroutput">bzip2</code> is built on top of
<code class="computeroutput">libbzip2</code>, a flexible library for
handling compressed data in the
<code class="computeroutput">bzip2</code> format. This manual
describes both how to use the program and how to work with the
library interface. Most of the manual is devoted to this
library, not the program, which is good news if your interest is
only in the program.</p>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc"><p><a class="xref" href="#using" title="2. How to use bzip2">How to use bzip2</a> describes how to use
<code class="computeroutput">bzip2</code>; this is the only part
you need to read if you just want to know how to operate the
program.</p></li>
<li class="listitem" style="list-style-type: disc"><p><a class="xref" href="#libprog" title="3. Programming with libbzip2">Programming with libbzip2</a> describes the
programming interfaces in detail, and</p></li>
<li class="listitem" style="list-style-type: disc"><p><a class="xref" href="#misc" title="4. Miscellanea">Miscellanea</a> records some
miscellaneous notes which I thought ought to be recorded
somewhere.</p></li>
</ul></div>
</div>
<div class="chapter" title="2. How to use bzip2">
<div class="titlepage"><div><div><h2 class="title">
<a name="using"></a>2. How to use bzip2</h2></div></div></div>
<div class="toc">
<p><b>Table of Contents</b></p>
<dl>
<dt><span class="sect1"><a href="#name">2.1. NAME</a></span></dt>
<dt><span class="sect1"><a href="#synopsis">2.2. SYNOPSIS</a></span></dt>
<dt><span class="sect1"><a href="#description">2.3. DESCRIPTION</a></span></dt>
<dt><span class="sect1"><a href="#options">2.4. OPTIONS</a></span></dt>
<dt><span class="sect1"><a href="#memory-management">2.5. MEMORY MANAGEMENT</a></span></dt>
<dt><span class="sect1"><a href="#recovering">2.6. RECOVERING DATA FROM DAMAGED FILES</a></span></dt>
<dt><span class="sect1"><a href="#performance">2.7. PERFORMANCE NOTES</a></span></dt>
<dt><span class="sect1"><a href="#caveats">2.8. CAVEATS</a></span></dt>
<dt><span class="sect1"><a href="#author">2.9. AUTHOR</a></span></dt>
</dl>
</div>
<p>This chapter contains a copy of the
<code class="computeroutput">bzip2</code> man page, and nothing
else.</p>
<div class="sect1" title="2.1. NAME">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="name"></a>2.1. NAME</h2></div></div></div>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2</code>,
<code class="computeroutput">bunzip2</code> - a block-sorting file
compressor, v1.0.6</p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzcat</code> -
decompresses files to stdout</p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2recover</code> -
recovers data from damaged bzip2 files</p></li>
</ul></div>
</div>
<div class="sect1" title="2.2. SYNOPSIS">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="synopsis"></a>2.2. SYNOPSIS</h2></div></div></div>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2</code> [
-cdfkqstvzVL123456789 ] [ filenames ... ]</p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bunzip2</code> [
-fkvsVL ] [ filenames ... ]</p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzcat</code> [ -s ] [
filenames ... ]</p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2recover</code>
filename</p></li>
</ul></div>
</div>
<div class="sect1" title="2.3. DESCRIPTION">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="description"></a>2.3. DESCRIPTION</h2></div></div></div>
<p><code class="computeroutput">bzip2</code> compresses files
using the Burrows-Wheeler block sorting text compression
algorithm, and Huffman coding. Compression is generally
considerably better than that achieved by more conventional
LZ77/LZ78-based compressors, and approaches the performance of
the PPM family of statistical compressors.</p>
<p>The command-line options are deliberately very similar to
those of GNU <code class="computeroutput">gzip</code>, but they are
not identical.</p>
<p><code class="computeroutput">bzip2</code> expects a list of
file names to accompany the command-line flags. Each file is
replaced by a compressed version of itself, with the name
<code class="computeroutput">original_name.bz2</code>. Each
compressed file has the same modification date, permissions, and,
when possible, ownership as the corresponding original, so that
these properties can be correctly restored at decompression time.
File name handling is naive in the sense that there is no
mechanism for preserving original file names, permissions,
ownerships or dates in filesystems which lack these concepts, or
have serious file name length restrictions, such as
MS-DOS.</p>
<p><code class="computeroutput">bzip2</code> and
<code class="computeroutput">bunzip2</code> will by default not
overwrite existing files. If you want this to happen, specify
the <code class="computeroutput">-f</code> flag.</p>
<p>If no file names are specified,
<code class="computeroutput">bzip2</code> compresses from standard
input to standard output. In this case,
<code class="computeroutput">bzip2</code> will decline to write
compressed output to a terminal, as this would be entirely
incomprehensible and therefore pointless.</p>
<p><code class="computeroutput">bunzip2</code> (or
<code class="computeroutput">bzip2 -d</code>) decompresses all
specified files. Files which were not created by
<code class="computeroutput">bzip2</code> will be detected and
ignored, and a warning issued.
<code class="computeroutput">bzip2</code> attempts to guess the
filename for the decompressed file from that of the compressed
file as follows:</p>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.bz2 </code>
becomes
<code class="computeroutput">filename</code></p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.bz </code>
becomes
<code class="computeroutput">filename</code></p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.tbz2</code>
becomes
<code class="computeroutput">filename.tar</code></p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.tbz </code>
becomes
<code class="computeroutput">filename.tar</code></p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">anyothername </code>
becomes
<code class="computeroutput">anyothername.out</code></p></li>
</ul></div>
<p>If the file does not end in one of the recognised endings,
<code class="computeroutput">.bz2</code>,
<code class="computeroutput">.bz</code>,
<code class="computeroutput">.tbz2</code> or
<code class="computeroutput">.tbz</code>,
<code class="computeroutput">bzip2</code> complains that it cannot
guess the name of the original file, and uses the original name
with <code class="computeroutput">.out</code> appended.</p>
<p>As with compression, supplying no filenames causes
decompression from standard input to standard output.</p>
<p><code class="computeroutput">bunzip2</code> will correctly
decompress a file which is the concatenation of two or more
compressed files. The result is the concatenation of the
corresponding uncompressed files. Integrity testing
(<code class="computeroutput">-t</code>) of concatenated compressed
files is also supported.</p>
<p>You can also compress or decompress files to the standard
output by giving the <code class="computeroutput">-c</code> flag.
Multiple files may be compressed and decompressed like this. The
resulting outputs are fed sequentially to stdout. Compression of
multiple files in this manner generates a stream containing
multiple compressed file representations. Such a stream can be
decompressed correctly only by
<code class="computeroutput">bzip2</code> version 0.9.0 or later.
Earlier versions of <code class="computeroutput">bzip2</code> will
stop after decompressing the first file in the stream.</p>
<p><code class="computeroutput">bzcat</code> (or
<code class="computeroutput">bzip2 -dc</code>) decompresses all
specified files to the standard output.</p>
<p><code class="computeroutput">bzip2</code> will read arguments
from the environment variables
<code class="computeroutput">BZIP2</code> and
<code class="computeroutput">BZIP</code>, in that order, and will
process them before any arguments read from the command line.
This gives a convenient way to supply default arguments.</p>
<p>Compression is always performed, even if the compressed
file is slightly larger than the original. Files of less than
about one hundred bytes tend to get larger, since the compression
mechanism has a constant overhead in the region of 50 bytes.
Random data (including the output of most file compressors) is
coded at about 8.05 bits per byte, giving an expansion of around
0.5%.</p>
<p>As a self-check for your protection,
<code class="computeroutput">bzip2</code> uses 32-bit CRCs to make
sure that the decompressed version of a file is identical to the
original. This guards against corruption of the compressed data,
and against undetected bugs in
<code class="computeroutput">bzip2</code> (hopefully very unlikely).
The chances of data corruption going undetected is microscopic,
about one chance in four billion for each file processed. Be
aware, though, that the check occurs upon decompression, so it
can only tell you that something is wrong. It can't help you
recover the original uncompressed data. You can use
<code class="computeroutput">bzip2recover</code> to try to recover
data from damaged files.</p>
<p>Return values: 0 for a normal exit, 1 for environmental
problems (file not found, invalid flags, I/O errors, etc.), 2
to indicate a corrupt compressed file, 3 for an internal
consistency error (eg, bug) which caused
<code class="computeroutput">bzip2</code> to panic.</p>
</div>
<div class="sect1" title="2.4. OPTIONS">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="options"></a>2.4. OPTIONS</h2></div></div></div>
<div class="variablelist"><dl>
<dt><span class="term"><code class="computeroutput">-c --stdout</code></span></dt>
<dd><p>Compress or decompress to standard
output.</p></dd>
<dt><span class="term"><code class="computeroutput">-d --decompress</code></span></dt>
<dd><p>Force decompression.
<code class="computeroutput">bzip2</code>,
<code class="computeroutput">bunzip2</code> and
<code class="computeroutput">bzcat</code> are really the same
program, and the decision about what actions to take is done on
the basis of which name is used. This flag overrides that
mechanism, and forces bzip2 to decompress.</p></dd>
<dt><span class="term"><code class="computeroutput">-z --compress</code></span></dt>
<dd><p>The complement to
<code class="computeroutput">-d</code>: forces compression,
regardless of the invokation name.</p></dd>
<dt><span class="term"><code class="computeroutput">-t --test</code></span></dt>
<dd><p>Check integrity of the specified file(s), but
don't decompress them. This really performs a trial
decompression and throws away the result.</p></dd>
<dt><span class="term"><code class="computeroutput">-f --force</code></span></dt>
<dd>
<p>Force overwrite of output files. Normally,
<code class="computeroutput">bzip2</code> will not overwrite
existing output files. Also forces
<code class="computeroutput">bzip2</code> to break hard links to
files, which it otherwise wouldn't do.</p>
<p><code class="computeroutput">bzip2</code> normally declines
to decompress files which don't have the correct magic header
bytes. If forced (<code class="computeroutput">-f</code>),
however, it will pass such files through unmodified. This is
how GNU <code class="computeroutput">gzip</code> behaves.</p>
</dd>
<dt><span class="term"><code class="computeroutput">-k --keep</code></span></dt>
<dd><p>Keep (don't delete) input files during
compression or decompression.</p></dd>
<dt><span class="term"><code class="computeroutput">-s --small</code></span></dt>
<dd>
<p>Reduce memory usage, for compression,
decompression and testing. Files are decompressed and tested
using a modified algorithm which only requires 2.5 bytes per
block byte. This means any file can be decompressed in 2300k
of memory, albeit at about half the normal speed.</p>
<p>During compression, <code class="computeroutput">-s</code>
selects a block size of 200k, which limits memory use to around
the same figure, at the expense of your compression ratio. In
short, if your machine is low on memory (8 megabytes or less),
use <code class="computeroutput">-s</code> for everything. See
<a class="xref" href="#memory-management" title="2.5. MEMORY MANAGEMENT">MEMORY MANAGEMENT</a> below.</p>
</dd>
<dt><span class="term"><code class="computeroutput">-q --quiet</code></span></dt>
<dd><p>Suppress non-essential warning messages.
Messages pertaining to I/O errors and other critical events
will not be suppressed.</p></dd>
<dt><span class="term"><code class="computeroutput">-v --verbose</code></span></dt>
<dd><p>Verbose mode -- show the compression ratio for
each file processed. Further
<code class="computeroutput">-v</code>'s increase the verbosity
level, spewing out lots of information which is primarily of
interest for diagnostic purposes.</p></dd>
<dt><span class="term"><code class="computeroutput">-L --license -V --version</code></span></dt>
<dd><p>Display the software version, license terms and
conditions.</p></dd>
<dt><span class="term"><code class="computeroutput">-1</code> (or
<code class="computeroutput">--fast</code>) to
<code class="computeroutput">-9</code> (or
<code class="computeroutput">-best</code>)</span></dt>
<dd><p>Set the block size to 100 k, 200 k ... 900 k
when compressing. Has no effect when decompressing. See <a class="xref" href="#memory-management" title="2.5. MEMORY MANAGEMENT">MEMORY MANAGEMENT</a> below. The
<code class="computeroutput">--fast</code> and
<code class="computeroutput">--best</code> aliases are primarily
for GNU <code class="computeroutput">gzip</code> compatibility.
In particular, <code class="computeroutput">--fast</code> doesn't
make things significantly faster. And
<code class="computeroutput">--best</code> merely selects the
default behaviour.</p></dd>
<dt><span class="term"><code class="computeroutput">--</code></span></dt>
<dd><p>Treats all subsequent arguments as file names,
even if they start with a dash. This is so you can handle
files with names beginning with a dash, for example:
<code class="computeroutput">bzip2 --
-myfilename</code>.</p></dd>
<dt>
<span class="term"><code class="computeroutput">--repetitive-fast</code>, </span><span class="term"><code class="computeroutput">--repetitive-best</code></span>
</dt>
<dd><p>These flags are redundant in versions 0.9.5 and
above. They provided some coarse control over the behaviour of
the sorting algorithm in earlier versions, which was sometimes
useful. 0.9.5 and above have an improved algorithm which
renders these flags irrelevant.</p></dd>
</dl></div>
</div>
<div class="sect1" title="2.5. MEMORY MANAGEMENT">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="memory-management"></a>2.5. MEMORY MANAGEMENT</h2></div></div></div>
<p><code class="computeroutput">bzip2</code> compresses large
files in blocks. The block size affects both the compression
ratio achieved, and the amount of memory needed for compression
and decompression. The flags <code class="computeroutput">-1</code>
through <code class="computeroutput">-9</code> specify the block
size to be 100,000 bytes through 900,000 bytes (the default)
respectively. At decompression time, the block size used for
compression is read from the header of the compressed file, and
<code class="computeroutput">bunzip2</code> then allocates itself
just enough memory to decompress the file. Since block sizes are
stored in compressed files, it follows that the flags
<code class="computeroutput">-1</code> to
<code class="computeroutput">-9</code> are irrelevant to and so
ignored during decompression.</p>
<p>Compression and decompression requirements, in bytes, can be
estimated as:</p>
<pre class="programlisting">Compression: 400k + ( 8 x block size )
Decompression: 100k + ( 4 x block size ), or
100k + ( 2.5 x block size )</pre>
<p>Larger block sizes give rapidly diminishing marginal
returns. Most of the compression comes from the first two or
three hundred k of block size, a fact worth bearing in mind when
using <code class="computeroutput">bzip2</code> on small machines.
It is also important to appreciate that the decompression memory
requirement is set at compression time by the choice of block
size.</p>
<p>For files compressed with the default 900k block size,
<code class="computeroutput">bunzip2</code> will require about 3700
kbytes to decompress. To support decompression of any file on a
4 megabyte machine, <code class="computeroutput">bunzip2</code> has
an option to decompress using approximately half this amount of
memory, about 2300 kbytes. Decompression speed is also halved,
so you should use this option only where necessary. The relevant
flag is <code class="computeroutput">-s</code>.</p>
<p>In general, try and use the largest block size memory
constraints allow, since that maximises the compression achieved.
Compression and decompression speed are virtually unaffected by
block size.</p>
<p>Another significant point applies to files which fit in a
single block -- that means most files you'd encounter using a
large block size. The amount of real memory touched is
proportional to the size of the file, since the file is smaller
than a block. For example, compressing a file 20,000 bytes long
with the flag <code class="computeroutput">-9</code> will cause the
compressor to allocate around 7600k of memory, but only touch
400k + 20000 * 8 = 560 kbytes of it. Similarly, the decompressor
will allocate 3700k but only touch 100k + 20000 * 4 = 180
kbytes.</p>
<p>Here is a table which summarises the maximum memory usage
for different block sizes. Also recorded is the total compressed
size for 14 files of the Calgary Text Compression Corpus
totalling 3,141,622 bytes. This column gives some feel for how
compression varies with block size. These figures tend to
understate the advantage of larger block sizes for larger files,
since the Corpus is dominated by smaller files.</p>
<pre class="programlisting"> Compress Decompress Decompress Corpus
Flag usage usage -s usage Size
-1 1200k 500k 350k 914704
-2 2000k 900k 600k 877703
-3 2800k 1300k 850k 860338
-4 3600k 1700k 1100k 846899
-5 4400k 2100k 1350k 845160
-6 5200k 2500k 1600k 838626
-7 6100k 2900k 1850k 834096
-8 6800k 3300k 2100k 828642
-9 7600k 3700k 2350k 828642</pre>
</div>
<div class="sect1" title="2.6. RECOVERING DATA FROM DAMAGED FILES">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="recovering"></a>2.6. RECOVERING DATA FROM DAMAGED FILES</h2></div></div></div>
<p><code class="computeroutput">bzip2</code> compresses files in
blocks, usually 900kbytes long. Each block is handled
independently. If a media or transmission error causes a
multi-block <code class="computeroutput">.bz2</code> file to become
damaged, it may be possible to recover data from the undamaged
blocks in the file.</p>
<p>The compressed representation of each block is delimited by
a 48-bit pattern, which makes it possible to find the block
boundaries with reasonable certainty. Each block also carries
its own 32-bit CRC, so damaged blocks can be distinguished from
undamaged ones.</p>
<p><code class="computeroutput">bzip2recover</code> is a simple
program whose purpose is to search for blocks in
<code class="computeroutput">.bz2</code> files, and write each block
out into its own <code class="computeroutput">.bz2</code> file. You
can then use <code class="computeroutput">bzip2 -t</code> to test
the integrity of the resulting files, and decompress those which
are undamaged.</p>
<p><code class="computeroutput">bzip2recover</code> takes a
single argument, the name of the damaged file, and writes a
number of files <code class="computeroutput">rec0001file.bz2</code>,
<code class="computeroutput">rec0002file.bz2</code>, etc, containing
the extracted blocks. The output filenames are designed so that
the use of wildcards in subsequent processing -- for example,
<code class="computeroutput">bzip2 -dc rec*file.bz2 >
recovered_data</code> -- lists the files in the correct
order.</p>
<p><code class="computeroutput">bzip2recover</code> should be of
most use dealing with large <code class="computeroutput">.bz2</code>
files, as these will contain many blocks. It is clearly futile
to use it on damaged single-block files, since a damaged block
cannot be recovered. If you wish to minimise any potential data
loss through media or transmission errors, you might consider
compressing with a smaller block size.</p>
</div>
<div class="sect1" title="2.7. PERFORMANCE NOTES">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="performance"></a>2.7. PERFORMANCE NOTES</h2></div></div></div>
<p>The sorting phase of compression gathers together similar
strings in the file. Because of this, files containing very long
runs of repeated symbols, like "aabaabaabaab ..." (repeated
several hundred times) may compress more slowly than normal.
Versions 0.9.5 and above fare much better than previous versions
in this respect. The ratio between worst-case and average-case
compression time is in the region of 10:1. For previous
versions, this figure was more like 100:1. You can use the
<code class="computeroutput">-vvvv</code> option to monitor progress
in great detail, if you want.</p>
<p>Decompression speed is unaffected by these
phenomena.</p>
<p><code class="computeroutput">bzip2</code> usually allocates
several megabytes of memory to operate in, and then charges all
over it in a fairly random fashion. This means that performance,
both for compressing and decompressing, is largely determined by
the speed at which your machine can service cache misses.
Because of this, small changes to the code to reduce the miss
rate have been observed to give disproportionately large
performance improvements. I imagine
<code class="computeroutput">bzip2</code> will perform best on
machines with very large caches.</p>
</div>
<div class="sect1" title="2.8. CAVEATS">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="caveats"></a>2.8. CAVEATS</h2></div></div></div>
<p>I/O error messages are not as helpful as they could be.
<code class="computeroutput">bzip2</code> tries hard to detect I/O
errors and exit cleanly, but the details of what the problem is
sometimes seem rather misleading.</p>
<p>This manual page pertains to version 1.0.6 of
<code class="computeroutput">bzip2</code>. Compressed data created by
this version is entirely forwards and backwards compatible with the
previous public releases, versions 0.1pl2, 0.9.0 and 0.9.5, 1.0.0,
1.0.1, 1.0.2 and 1.0.3, but with the following exception: 0.9.0 and
above can correctly decompress multiple concatenated compressed files.
0.1pl2 cannot do this; it will stop after decompressing just the first
file in the stream.</p>
<p><code class="computeroutput">bzip2recover</code> versions
prior to 1.0.2 used 32-bit integers to represent bit positions in
compressed files, so it could not handle compressed files more
than 512 megabytes long. Versions 1.0.2 and above use 64-bit ints
on some platforms which support them (GNU supported targets, and
Windows). To establish whether or not
<code class="computeroutput">bzip2recover</code> was built with such
a limitation, run it without arguments. In any event you can
build yourself an unlimited version if you can recompile it with
<code class="computeroutput">MaybeUInt64</code> set to be an
unsigned 64-bit integer.</p>
</div>
<div class="sect1" title="2.9. AUTHOR">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="author"></a>2.9. AUTHOR</h2></div></div></div>
<p>Julian Seward,
<code class="computeroutput">jseward@bzip.org</code></p>
<p>The ideas embodied in
<code class="computeroutput">bzip2</code> are due to (at least) the
following people: Michael Burrows and David Wheeler (for the
block sorting transformation), David Wheeler (again, for the
Huffman coder), Peter Fenwick (for the structured coding model in
the original <code class="computeroutput">bzip</code>, and many
refinements), and Alistair Moffat, Radford Neal and Ian Witten
(for the arithmetic coder in the original
<code class="computeroutput">bzip</code>). I am much indebted for
their help, support and advice. See the manual in the source
distribution for pointers to sources of documentation. Christian
von Roques encouraged me to look for faster sorting algorithms,
so as to speed up compression. Bela Lubkin encouraged me to
improve the worst-case compression performance.
Donna Robinson XMLised the documentation.
Many people sent
patches, helped with portability problems, lent machines, gave
advice and were generally helpful.</p>
</div>
</div>
<div class="chapter" title="3. Programming with libbzip2">
<div class="titlepage"><div><div><h2 class="title">
<a name="libprog"></a>3.
Programming with <code class="computeroutput">libbzip2</code>
</h2></div></div></div>
<div class="toc">
<p><b>Table of Contents</b></p>
<dl>
<dt><span class="sect1"><a href="#top-level">3.1. Top-level structure</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#ll-summary">3.1.1. Low-level summary</a></span></dt>
<dt><span class="sect2"><a href="#hl-summary">3.1.2. High-level summary</a></span></dt>
<dt><span class="sect2"><a href="#util-fns-summary">3.1.3. Utility functions summary</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#err-handling">3.2. Error handling</a></span></dt>
<dt><span class="sect1"><a href="#low-level">3.3. Low-level interface</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#bzcompress-init">3.3.1. BZ2_bzCompressInit</a></span></dt>
<dt><span class="sect2"><a href="#bzCompress">3.3.2. BZ2_bzCompress</a></span></dt>
<dt><span class="sect2"><a href="#bzCompress-end">3.3.3. BZ2_bzCompressEnd</a></span></dt>
<dt><span class="sect2"><a href="#bzDecompress-init">3.3.4. BZ2_bzDecompressInit</a></span></dt>
<dt><span class="sect2"><a href="#bzDecompress">3.3.5. BZ2_bzDecompress</a></span></dt>
<dt><span class="sect2"><a href="#bzDecompress-end">3.3.6. BZ2_bzDecompressEnd</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#hl-interface">3.4. High-level interface</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#bzreadopen">3.4.1. BZ2_bzReadOpen</a></span></dt>
<dt><span class="sect2"><a href="#bzread">3.4.2. BZ2_bzRead</a></span></dt>
<dt><span class="sect2"><a href="#bzreadgetunused">3.4.3. BZ2_bzReadGetUnused</a></span></dt>
<dt><span class="sect2"><a href="#bzreadclose">3.4.4. BZ2_bzReadClose</a></span></dt>
<dt><span class="sect2"><a href="#bzwriteopen">3.4.5. BZ2_bzWriteOpen</a></span></dt>
<dt><span class="sect2"><a href="#bzwrite">3.4.6. BZ2_bzWrite</a></span></dt>
<dt><span class="sect2"><a href="#bzwriteclose">3.4.7. BZ2_bzWriteClose</a></span></dt>
<dt><span class="sect2"><a href="#embed">3.4.8. Handling embedded compressed data streams</a></span></dt>
<dt><span class="sect2"><a href="#std-rdwr">3.4.9. Standard file-reading/writing code</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#util-fns">3.5. Utility functions</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#bzbufftobuffcompress">3.5.1. BZ2_bzBuffToBuffCompress</a></span></dt>
<dt><span class="sect2"><a href="#bzbufftobuffdecompress">3.5.2. BZ2_bzBuffToBuffDecompress</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#zlib-compat">3.6. zlib compatibility functions</a></span></dt>
<dt><span class="sect1"><a href="#stdio-free">3.7. Using the library in a stdio-free environment</a></span></dt>
<dd><dl>
<dt><span class="sect2"><a href="#stdio-bye">3.7.1. Getting rid of stdio</a></span></dt>
<dt><span class="sect2"><a href="#critical-error">3.7.2. Critical error handling</a></span></dt>
</dl></dd>
<dt><span class="sect1"><a href="#win-dll">3.8. Making a Windows DLL</a></span></dt>
</dl>
</div>
<p>This chapter describes the programming interface to
<code class="computeroutput">libbzip2</code>.</p>
<p>For general background information, particularly about
memory use and performance aspects, you'd be well advised to read
<a class="xref" href="#using" title="2. How to use bzip2">How to use bzip2</a> as well.</p>
<div class="sect1" title="3.1. Top-level structure">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="top-level"></a>3.1. Top-level structure</h2></div></div></div>
<p><code class="computeroutput">libbzip2</code> is a flexible
library for compressing and decompressing data in the
<code class="computeroutput">bzip2</code> data format. Although
packaged as a single entity, it helps to regard the library as
three separate parts: the low level interface, and the high level
interface, and some utility functions.</p>
<p>The structure of
<code class="computeroutput">libbzip2</code>'s interfaces is similar
to that of Jean-loup Gailly's and Mark Adler's excellent
<code class="computeroutput">zlib</code> library.</p>
<p>All externally visible symbols have names beginning
<code class="computeroutput">BZ2_</code>. This is new in version
1.0. The intention is to minimise pollution of the namespaces of
library clients.</p>
<p>To use any part of the library, you need to
<code class="computeroutput">#include <bzlib.h></code>
into your sources.</p>
<div class="sect2" title="3.1.1. Low-level summary">
<div class="titlepage"><div><div><h3 class="title">
<a name="ll-summary"></a>3.1.1. Low-level summary</h3></div></div></div>
<p>This interface provides services for compressing and
decompressing data in memory. There's no provision for dealing
with files, streams or any other I/O mechanisms, just straight
memory-to-memory work. In fact, this part of the library can be
compiled without inclusion of
<code class="computeroutput">stdio.h</code>, which may be helpful
for embedded applications.</p>
<p>The low-level part of the library has no global variables
and is therefore thread-safe.</p>
<p>Six routines make up the low level interface:
<code class="computeroutput">BZ2_bzCompressInit</code>,
<code class="computeroutput">BZ2_bzCompress</code>, and
<code class="computeroutput">BZ2_bzCompressEnd</code> for
compression, and a corresponding trio
<code class="computeroutput">BZ2_bzDecompressInit</code>,
<code class="computeroutput">BZ2_bzDecompress</code> and
<code class="computeroutput">BZ2_bzDecompressEnd</code> for
decompression. The <code class="computeroutput">*Init</code>
functions allocate memory for compression/decompression and do
other initialisations, whilst the
<code class="computeroutput">*End</code> functions close down
operations and release memory.</p>
<p>The real work is done by
<code class="computeroutput">BZ2_bzCompress</code> and
<code class="computeroutput">BZ2_bzDecompress</code>. These
compress and decompress data from a user-supplied input buffer to
a user-supplied output buffer. These buffers can be any size;
arbitrary quantities of data are handled by making repeated calls
to these functions. This is a flexible mechanism allowing a
consumer-pull style of activity, or producer-push, or a mixture
of both.</p>
</div>
<div class="sect2" title="3.1.2. High-level summary">
<div class="titlepage"><div><div><h3 class="title">
<a name="hl-summary"></a>3.1.2. High-level summary</h3></div></div></div>
<p>This interface provides some handy wrappers around the
low-level interface to facilitate reading and writing
<code class="computeroutput">bzip2</code> format files
(<code class="computeroutput">.bz2</code> files). The routines
provide hooks to facilitate reading files in which the
<code class="computeroutput">bzip2</code> data stream is embedded
within some larger-scale file structure, or where there are
multiple <code class="computeroutput">bzip2</code> data streams
concatenated end-to-end.</p>
<p>For reading files,
<code class="computeroutput">BZ2_bzReadOpen</code>,
<code class="computeroutput">BZ2_bzRead</code>,
<code class="computeroutput">BZ2_bzReadClose</code> and
<code class="computeroutput">BZ2_bzReadGetUnused</code> are
supplied. For writing files,
<code class="computeroutput">BZ2_bzWriteOpen</code>,
<code class="computeroutput">BZ2_bzWrite</code> and
<code class="computeroutput">BZ2_bzWriteFinish</code> are
available.</p>
<p>As with the low-level library, no global variables are used
so the library is per se thread-safe. However, if I/O errors
occur whilst reading or writing the underlying compressed files,
you may have to consult <code class="computeroutput">errno</code> to
determine the cause of the error. In that case, you'd need a C
library which correctly supports
<code class="computeroutput">errno</code> in a multithreaded
environment.</p>
<p>To make the library a little simpler and more portable,
<code class="computeroutput">BZ2_bzReadOpen</code> and
<code class="computeroutput">BZ2_bzWriteOpen</code> require you to
pass them file handles (<code class="computeroutput">FILE*</code>s)
which have previously been opened for reading or writing
respectively. That avoids portability problems associated with
file operations and file attributes, whilst not being much of an
imposition on the programmer.</p>
</div>
<div class="sect2" title="3.1.3. Utility functions summary">
<div class="titlepage"><div><div><h3 class="title">
<a name="util-fns-summary"></a>3.1.3. Utility functions summary</h3></div></div></div>
<p>For very simple needs,
<code class="computeroutput">BZ2_bzBuffToBuffCompress</code> and
<code class="computeroutput">BZ2_bzBuffToBuffDecompress</code> are
provided. These compress data in memory from one buffer to
another buffer in a single function call. You should assess
whether these functions fulfill your memory-to-memory
compression/decompression requirements before investing effort in
understanding the more general but more complex low-level
interface.</p>
<p>Yoshioka Tsuneo
(<code class="computeroutput">tsuneo@rr.iij4u.or.jp</code>) has
contributed some functions to give better
<code class="computeroutput">zlib</code> compatibility. These
functions are <code class="computeroutput">BZ2_bzopen</code>,
<code class="computeroutput">BZ2_bzread</code>,
<code class="computeroutput">BZ2_bzwrite</code>,
<code class="computeroutput">BZ2_bzflush</code>,
<code class="computeroutput">BZ2_bzclose</code>,
<code class="computeroutput">BZ2_bzerror</code> and
<code class="computeroutput">BZ2_bzlibVersion</code>. You may find
these functions more convenient for simple file reading and
writing, than those in the high-level interface. These functions
are not (yet) officially part of the library, and are minimally
documented here. If they break, you get to keep all the pieces.
I hope to document them properly when time permits.</p>
<p>Yoshioka also contributed modifications to allow the
library to be built as a Windows DLL.</p>
</div>
</div>
<div class="sect1" title="3.2. Error handling">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="err-handling"></a>3.2. Error handling</h2></div></div></div>
<p>The library is designed to recover cleanly in all
situations, including the worst-case situation of decompressing
random data. I'm not 100% sure that it can always do this, so
you might want to add a signal handler to catch segmentation
violations during decompression if you are feeling especially
paranoid. I would be interested in hearing more about the
robustness of the library to corrupted compressed data.</p>
<p>Version 1.0.3 more robust in this respect than any
previous version. Investigations with Valgrind (a tool for detecting
problems with memory management) indicate
that, at least for the few files I tested, all single-bit errors
in the decompressed data are caught properly, with no
segmentation faults, no uses of uninitialised data, no out of
range reads or writes, and no infinite looping in the decompressor.
So it's certainly pretty robust, although
I wouldn't claim it to be totally bombproof.</p>
<p>The file <code class="computeroutput">bzlib.h</code> contains
all definitions needed to use the library. In particular, you
should definitely not include
<code class="computeroutput">bzlib_private.h</code>.</p>
<p>In <code class="computeroutput">bzlib.h</code>, the various
return values are defined. The following list is not intended as
an exhaustive description of the circumstances in which a given
value may be returned -- those descriptions are given later.
Rather, it is intended to convey the rough meaning of each return
value. The first five actions are normal and not intended to
denote an error situation.</p>
<div class="variablelist"><dl>
<dt><span class="term"><code class="computeroutput">BZ_OK</code></span></dt>
<dd><p>The requested action was completed
successfully.</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_RUN_OK, BZ_FLUSH_OK,
BZ_FINISH_OK</code></span></dt>
<dd><p>In
<code class="computeroutput">BZ2_bzCompress</code>, the requested
flush/finish/nothing-special action was completed
successfully.</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_STREAM_END</code></span></dt>
<dd><p>Compression of data was completed, or the
logical stream end was detected during
decompression.</p></dd>
</dl></div>
<p>The following return values indicate an error of some
kind.</p>
<div class="variablelist"><dl>
<dt><span class="term"><code class="computeroutput">BZ_CONFIG_ERROR</code></span></dt>
<dd><p>Indicates that the library has been improperly
compiled on your platform -- a major configuration error.
Specifically, it means that
<code class="computeroutput">sizeof(char)</code>,
<code class="computeroutput">sizeof(short)</code> and
<code class="computeroutput">sizeof(int)</code> are not 1, 2 and
4 respectively, as they should be. Note that the library
should still work properly on 64-bit platforms which follow
the LP64 programming model -- that is, where
<code class="computeroutput">sizeof(long)</code> and
<code class="computeroutput">sizeof(void*)</code> are 8. Under
LP64, <code class="computeroutput">sizeof(int)</code> is still 4,
so <code class="computeroutput">libbzip2</code>, which doesn't
use the <code class="computeroutput">long</code> type, is
OK.</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_SEQUENCE_ERROR</code></span></dt>
<dd><p>When using the library, it is important to call
the functions in the correct sequence and with data structures
(buffers etc) in the correct states.
<code class="computeroutput">libbzip2</code> checks as much as it
can to ensure this is happening, and returns
<code class="computeroutput">BZ_SEQUENCE_ERROR</code> if not.
Code which complies precisely with the function semantics, as
detailed below, should never receive this value; such an event
denotes buggy code which you should
investigate.</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_PARAM_ERROR</code></span></dt>
<dd><p>Returned when a parameter to a function call is
out of range or otherwise manifestly incorrect. As with
<code class="computeroutput">BZ_SEQUENCE_ERROR</code>, this
denotes a bug in the client code. The distinction between
<code class="computeroutput">BZ_PARAM_ERROR</code> and
<code class="computeroutput">BZ_SEQUENCE_ERROR</code> is a bit
hazy, but still worth making.</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_MEM_ERROR</code></span></dt>
<dd><p>Returned when a request to allocate memory
failed. Note that the quantity of memory needed to decompress
a stream cannot be determined until the stream's header has
been read. So
<code class="computeroutput">BZ2_bzDecompress</code> and
<code class="computeroutput">BZ2_bzRead</code> may return
<code class="computeroutput">BZ_MEM_ERROR</code> even though some
of the compressed data has been read. The same is not true
for compression; once
<code class="computeroutput">BZ2_bzCompressInit</code> or
<code class="computeroutput">BZ2_bzWriteOpen</code> have
successfully completed,
<code class="computeroutput">BZ_MEM_ERROR</code> cannot
occur.</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_DATA_ERROR</code></span></dt>
<dd><p>Returned when a data integrity error is
detected during decompression. Most importantly, this means
when stored and computed CRCs for the data do not match. This
value is also returned upon detection of any other anomaly in
the compressed data.</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_DATA_ERROR_MAGIC</code></span></dt>
<dd><p>As a special case of
<code class="computeroutput">BZ_DATA_ERROR</code>, it is
sometimes useful to know when the compressed stream does not
start with the correct magic bytes (<code class="computeroutput">'B' 'Z'
'h'</code>).</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_IO_ERROR</code></span></dt>
<dd><p>Returned by
<code class="computeroutput">BZ2_bzRead</code> and
<code class="computeroutput">BZ2_bzWrite</code> when there is an
error reading or writing in the compressed file, and by
<code class="computeroutput">BZ2_bzReadOpen</code> and
<code class="computeroutput">BZ2_bzWriteOpen</code> for attempts
to use a file for which the error indicator (viz,
<code class="computeroutput">ferror(f)</code>) is set. On
receipt of <code class="computeroutput">BZ_IO_ERROR</code>, the
caller should consult <code class="computeroutput">errno</code>
and/or <code class="computeroutput">perror</code> to acquire
operating-system specific information about the
problem.</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_UNEXPECTED_EOF</code></span></dt>
<dd><p>Returned by
<code class="computeroutput">BZ2_bzRead</code> when the
compressed file finishes before the logical end of stream is
detected.</p></dd>
<dt><span class="term"><code class="computeroutput">BZ_OUTBUFF_FULL</code></span></dt>
<dd><p>Returned by
<code class="computeroutput">BZ2_bzBuffToBuffCompress</code> and
<code class="computeroutput">BZ2_bzBuffToBuffDecompress</code> to
indicate that the output data will not fit into the output
buffer provided.</p></dd>
</dl></div>
</div>
<div class="sect1" title="3.3. Low-level interface">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="low-level"></a>3.3. Low-level interface</h2></div></div></div>
<div class="sect2" title="3.3.1. BZ2_bzCompressInit">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzcompress-init"></a>3.3.1. BZ2_bzCompressInit</h3></div></div></div>
<pre class="programlisting">typedef struct {
char *next_in;
unsigned int avail_in;
unsigned int total_in_lo32;
unsigned int total_in_hi32;
char *next_out;
unsigned int avail_out;
unsigned int total_out_lo32;
unsigned int total_out_hi32;
void *state;
void *(*bzalloc)(void *,int,int);
void (*bzfree)(void *,void *);
void *opaque;
} bz_stream;
int BZ2_bzCompressInit ( bz_stream *strm,
int blockSize100k,
int verbosity,
int workFactor );</pre>
<p>Prepares for compression. The
<code class="computeroutput">bz_stream</code> structure holds all
data pertaining to the compression activity. A
<code class="computeroutput">bz_stream</code> structure should be
allocated and initialised prior to the call. The fields of
<code class="computeroutput">bz_stream</code> comprise the entirety
of the user-visible data. <code class="computeroutput">state</code>
is a pointer to the private data structures required for
compression.</p>
<p>Custom memory allocators are supported, via fields
<code class="computeroutput">bzalloc</code>,
<code class="computeroutput">bzfree</code>, and
<code class="computeroutput">opaque</code>. The value
<code class="computeroutput">opaque</code> is passed to as the first
argument to all calls to <code class="computeroutput">bzalloc</code>
and <code class="computeroutput">bzfree</code>, but is otherwise
ignored by the library. The call <code class="computeroutput">bzalloc (
opaque, n, m )</code> is expected to return a pointer
<code class="computeroutput">p</code> to <code class="computeroutput">n *
m</code> bytes of memory, and <code class="computeroutput">bzfree (
opaque, p )</code> should free that memory.</p>
<p>If you don't want to use a custom memory allocator, set
<code class="computeroutput">bzalloc</code>,
<code class="computeroutput">bzfree</code> and
<code class="computeroutput">opaque</code> to
<code class="computeroutput">NULL</code>, and the library will then
use the standard <code class="computeroutput">malloc</code> /
<code class="computeroutput">free</code> routines.</p>
<p>Before calling
<code class="computeroutput">BZ2_bzCompressInit</code>, fields
<code class="computeroutput">bzalloc</code>,
<code class="computeroutput">bzfree</code> and
<code class="computeroutput">opaque</code> should be filled
appropriately, as just described. Upon return, the internal
state will have been allocated and initialised, and
<code class="computeroutput">total_in_lo32</code>,
<code class="computeroutput">total_in_hi32</code>,
<code class="computeroutput">total_out_lo32</code> and
<code class="computeroutput">total_out_hi32</code> will have been
set to zero. These four fields are used by the library to inform
the caller of the total amount of data passed into and out of the
library, respectively. You should not try to change them. As of
version 1.0, 64-bit counts are maintained, even on 32-bit
platforms, using the <code class="computeroutput">_hi32</code>
fields to store the upper 32 bits of the count. So, for example,
the total amount of data in is <code class="computeroutput">(total_in_hi32
<< 32) + total_in_lo32</code>.</p>
<p>Parameter <code class="computeroutput">blockSize100k</code>
specifies the block size to be used for compression. It should
be a value between 1 and 9 inclusive, and the actual block size
used is 100000 x this figure. 9 gives the best compression but
takes most memory.</p>
<p>Parameter <code class="computeroutput">verbosity</code> should
be set to a number between 0 and 4 inclusive. 0 is silent, and
greater numbers give increasingly verbose monitoring/debugging
output. If the library has been compiled with
<code class="computeroutput">-DBZ_NO_STDIO</code>, no such output
will appear for any verbosity setting.</p>
<p>Parameter <code class="computeroutput">workFactor</code>
controls how the compression phase behaves when presented with
worst case, highly repetitive, input data. If compression runs
into difficulties caused by repetitive data, the library switches
from the standard sorting algorithm to a fallback algorithm. The
fallback is slower than the standard algorithm by perhaps a
factor of three, but always behaves reasonably, no matter how bad
the input.</p>
<p>Lower values of <code class="computeroutput">workFactor</code>
reduce the amount of effort the standard algorithm will expend
before resorting to the fallback. You should set this parameter
carefully; too low, and many inputs will be handled by the
fallback algorithm and so compress rather slowly, too high, and
your average-to-worst case compression times can become very
large. The default value of 30 gives reasonable behaviour over a
wide range of circumstances.</p>
<p>Allowable values range from 0 to 250 inclusive. 0 is a
special case, equivalent to using the default value of 30.</p>
<p>Note that the compressed output generated is the same
regardless of whether or not the fallback algorithm is
used.</p>
<p>Be aware also that this parameter may disappear entirely in
future versions of the library. In principle it should be
possible to devise a good way to automatically choose which
algorithm to use. Such a mechanism would render the parameter
obsolete.</p>
<p>Possible return values:</p>
<pre class="programlisting">BZ_CONFIG_ERROR
if the library has been mis-compiled
BZ_PARAM_ERROR
if strm is NULL
or blockSize < 1 or blockSize > 9
or verbosity < 0 or verbosity > 4
or workFactor < 0 or workFactor > 250
BZ_MEM_ERROR
if not enough memory is available
BZ_OK
otherwise</pre>
<p>Allowable next actions:</p>
<pre class="programlisting">BZ2_bzCompress
if BZ_OK is returned
no specific action needed in case of error</pre>
</div>
<div class="sect2" title="3.3.2. BZ2_bzCompress">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzCompress"></a>3.3.2. BZ2_bzCompress</h3></div></div></div>
<pre class="programlisting">int BZ2_bzCompress ( bz_stream *strm, int action );</pre>
<p>Provides more input and/or output buffer space for the
library. The caller maintains input and output buffers, and
calls <code class="computeroutput">BZ2_bzCompress</code> to transfer
data between them.</p>
<p>Before each call to
<code class="computeroutput">BZ2_bzCompress</code>,
<code class="computeroutput">next_in</code> should point at the data
to be compressed, and <code class="computeroutput">avail_in</code>
should indicate how many bytes the library may read.
<code class="computeroutput">BZ2_bzCompress</code> updates
<code class="computeroutput">next_in</code>,
<code class="computeroutput">avail_in</code> and
<code class="computeroutput">total_in</code> to reflect the number
of bytes it has read.</p>
<p>Similarly, <code class="computeroutput">next_out</code> should
point to a buffer in which the compressed data is to be placed,
with <code class="computeroutput">avail_out</code> indicating how
much output space is available.
<code class="computeroutput">BZ2_bzCompress</code> updates
<code class="computeroutput">next_out</code>,
<code class="computeroutput">avail_out</code> and
<code class="computeroutput">total_out</code> to reflect the number
of bytes output.</p>
<p>You may provide and remove as little or as much data as you
like on each call of
<code class="computeroutput">BZ2_bzCompress</code>. In the limit,
it is acceptable to supply and remove data one byte at a time,
although this would be terribly inefficient. You should always
ensure that at least one byte of output space is available at
each call.</p>
<p>A second purpose of
<code class="computeroutput">BZ2_bzCompress</code> is to request a
change of mode of the compressed stream.</p>
<p>Conceptually, a compressed stream can be in one of four
states: IDLE, RUNNING, FLUSHING and FINISHING. Before
initialisation
(<code class="computeroutput">BZ2_bzCompressInit</code>) and after
termination (<code class="computeroutput">BZ2_bzCompressEnd</code>),
a stream is regarded as IDLE.</p>
<p>Upon initialisation
(<code class="computeroutput">BZ2_bzCompressInit</code>), the stream
is placed in the RUNNING state. Subsequent calls to
<code class="computeroutput">BZ2_bzCompress</code> should pass
<code class="computeroutput">BZ_RUN</code> as the requested action;
other actions are illegal and will result in
<code class="computeroutput">BZ_SEQUENCE_ERROR</code>.</p>
<p>At some point, the calling program will have provided all
the input data it wants to. It will then want to finish up -- in
effect, asking the library to process any data it might have
buffered internally. In this state,
<code class="computeroutput">BZ2_bzCompress</code> will no longer
attempt to read data from
<code class="computeroutput">next_in</code>, but it will want to
write data to <code class="computeroutput">next_out</code>. Because
the output buffer supplied by the user can be arbitrarily small,
the finishing-up operation cannot necessarily be done with a
single call of
<code class="computeroutput">BZ2_bzCompress</code>.</p>
<p>Instead, the calling program passes
<code class="computeroutput">BZ_FINISH</code> as an action to
<code class="computeroutput">BZ2_bzCompress</code>. This changes
the stream's state to FINISHING. Any remaining input (ie,
<code class="computeroutput">next_in[0 .. avail_in-1]</code>) is
compressed and transferred to the output buffer. To do this,
<code class="computeroutput">BZ2_bzCompress</code> must be called
repeatedly until all the output has been consumed. At that
point, <code class="computeroutput">BZ2_bzCompress</code> returns
<code class="computeroutput">BZ_STREAM_END</code>, and the stream's
state is set back to IDLE.
<code class="computeroutput">BZ2_bzCompressEnd</code> should then be
called.</p>
<p>Just to make sure the calling program does not cheat, the
library makes a note of <code class="computeroutput">avail_in</code>
at the time of the first call to
<code class="computeroutput">BZ2_bzCompress</code> which has
<code class="computeroutput">BZ_FINISH</code> as an action (ie, at
the time the program has announced its intention to not supply
any more input). By comparing this value with that of
<code class="computeroutput">avail_in</code> over subsequent calls
to <code class="computeroutput">BZ2_bzCompress</code>, the library
can detect any attempts to slip in more data to compress. Any
calls for which this is detected will return
<code class="computeroutput">BZ_SEQUENCE_ERROR</code>. This
indicates a programming mistake which should be corrected.</p>
<p>Instead of asking to finish, the calling program may ask
<code class="computeroutput">BZ2_bzCompress</code> to take all the
remaining input, compress it and terminate the current
(Burrows-Wheeler) compression block. This could be useful for
error control purposes. The mechanism is analogous to that for
finishing: call <code class="computeroutput">BZ2_bzCompress</code>
with an action of <code class="computeroutput">BZ_FLUSH</code>,
remove output data, and persist with the
<code class="computeroutput">BZ_FLUSH</code> action until the value
<code class="computeroutput">BZ_RUN</code> is returned. As with
finishing, <code class="computeroutput">BZ2_bzCompress</code>
detects any attempt to provide more input data once the flush has
begun.</p>
<p>Once the flush is complete, the stream returns to the
normal RUNNING state.</p>
<p>This all sounds pretty complex, but isn't really. Here's a
table which shows which actions are allowable in each state, what
action will be taken, what the next state is, and what the
non-error return values are. Note that you can't explicitly ask
what state the stream is in, but nor do you need to -- it can be
inferred from the values returned by
<code class="computeroutput">BZ2_bzCompress</code>.</p>
<pre class="programlisting">IDLE/any
Illegal. IDLE state only exists after BZ2_bzCompressEnd or
before BZ2_bzCompressInit.
Return value = BZ_SEQUENCE_ERROR
RUNNING/BZ_RUN
Compress from next_in to next_out as much as possible.
Next state = RUNNING
Return value = BZ_RUN_OK
RUNNING/BZ_FLUSH
Remember current value of next_in. Compress from next_in
to next_out as much as possible, but do not accept any more input.
Next state = FLUSHING
Return value = BZ_FLUSH_OK
RUNNING/BZ_FINISH
Remember current value of next_in. Compress from next_in
to next_out as much as possible, but do not accept any more input.
Next state = FINISHING
Return value = BZ_FINISH_OK
FLUSHING/BZ_FLUSH
Compress from next_in to next_out as much as possible,
but do not accept any more input.
If all the existing input has been used up and all compressed
output has been removed
Next state = RUNNING; Return value = BZ_RUN_OK
else
Next state = FLUSHING; Return value = BZ_FLUSH_OK
FLUSHING/other
Illegal.
Return value = BZ_SEQUENCE_ERROR
FINISHING/BZ_FINISH
Compress from next_in to next_out as much as possible,
but to not accept any more input.
If all the existing input has been used up and all compressed
output has been removed
Next state = IDLE; Return value = BZ_STREAM_END
else
Next state = FINISHING; Return value = BZ_FINISH_OK
FINISHING/other
Illegal.
Return value = BZ_SEQUENCE_ERROR</pre>
<p>That still looks complicated? Well, fair enough. The
usual sequence of calls for compressing a load of data is:</p>
<div class="orderedlist"><ol class="orderedlist" type="1">
<li class="listitem"><p>Get started with
<code class="computeroutput">BZ2_bzCompressInit</code>.</p></li>
<li class="listitem"><p>Shovel data in and shlurp out its compressed form
using zero or more calls of
<code class="computeroutput">BZ2_bzCompress</code> with action =
<code class="computeroutput">BZ_RUN</code>.</p></li>
<li class="listitem"><p>Finish up. Repeatedly call
<code class="computeroutput">BZ2_bzCompress</code> with action =
<code class="computeroutput">BZ_FINISH</code>, copying out the
compressed output, until
<code class="computeroutput">BZ_STREAM_END</code> is
returned.</p></li>
<li class="listitem"><p>Close up and go home. Call
<code class="computeroutput">BZ2_bzCompressEnd</code>.</p></li>
</ol></div>
<p>If the data you want to compress fits into your input
buffer all at once, you can skip the calls of
<code class="computeroutput">BZ2_bzCompress ( ..., BZ_RUN )</code>
and just do the <code class="computeroutput">BZ2_bzCompress ( ..., BZ_FINISH
)</code> calls.</p>
<p>All required memory is allocated by
<code class="computeroutput">BZ2_bzCompressInit</code>. The
compression library can accept any data at all (obviously). So
you shouldn't get any error return values from the
<code class="computeroutput">BZ2_bzCompress</code> calls. If you
do, they will be
<code class="computeroutput">BZ_SEQUENCE_ERROR</code>, and indicate
a bug in your programming.</p>
<p>Trivial other possible return values:</p>
<pre class="programlisting">BZ_PARAM_ERROR
if strm is NULL, or strm->s is NULL</pre>
</div>
<div class="sect2" title="3.3.3. BZ2_bzCompressEnd">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzCompress-end"></a>3.3.3. BZ2_bzCompressEnd</h3></div></div></div>
<pre class="programlisting">int BZ2_bzCompressEnd ( bz_stream *strm );</pre>
<p>Releases all memory associated with a compression
stream.</p>
<p>Possible return values:</p>
<pre class="programlisting">BZ_PARAM_ERROR if strm is NULL or strm->s is NULL
BZ_OK otherwise</pre>
</div>
<div class="sect2" title="3.3.4. BZ2_bzDecompressInit">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzDecompress-init"></a>3.3.4. BZ2_bzDecompressInit</h3></div></div></div>
<pre class="programlisting">int BZ2_bzDecompressInit ( bz_stream *strm, int verbosity, int small );</pre>
<p>Prepares for decompression. As with
<code class="computeroutput">BZ2_bzCompressInit</code>, a
<code class="computeroutput">bz_stream</code> record should be
allocated and initialised before the call. Fields
<code class="computeroutput">bzalloc</code>,
<code class="computeroutput">bzfree</code> and
<code class="computeroutput">opaque</code> should be set if a custom
memory allocator is required, or made
<code class="computeroutput">NULL</code> for the normal
<code class="computeroutput">malloc</code> /
<code class="computeroutput">free</code> routines. Upon return, the
internal state will have been initialised, and
<code class="computeroutput">total_in</code> and
<code class="computeroutput">total_out</code> will be zero.</p>
<p>For the meaning of parameter
<code class="computeroutput">verbosity</code>, see
<code class="computeroutput">BZ2_bzCompressInit</code>.</p>
<p>If <code class="computeroutput">small</code> is nonzero, the
library will use an alternative decompression algorithm which
uses less memory but at the cost of decompressing more slowly
(roughly speaking, half the speed, but the maximum memory
requirement drops to around 2300k). See <a class="xref" href="#using" title="2. How to use bzip2">How to use bzip2</a>
for more information on memory management.</p>
<p>Note that the amount of memory needed to decompress a
stream cannot be determined until the stream's header has been
read, so even if
<code class="computeroutput">BZ2_bzDecompressInit</code> succeeds, a
subsequent <code class="computeroutput">BZ2_bzDecompress</code>
could fail with
<code class="computeroutput">BZ_MEM_ERROR</code>.</p>
<p>Possible return values:</p>
<pre class="programlisting">BZ_CONFIG_ERROR
if the library has been mis-compiled
BZ_PARAM_ERROR
if ( small != 0 && small != 1 )
or (verbosity <; 0 || verbosity > 4)
BZ_MEM_ERROR
if insufficient memory is available</pre>
<p>Allowable next actions:</p>
<pre class="programlisting">BZ2_bzDecompress
if BZ_OK was returned
no specific action required in case of error</pre>
</div>
<div class="sect2" title="3.3.5. BZ2_bzDecompress">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzDecompress"></a>3.3.5. BZ2_bzDecompress</h3></div></div></div>
<pre class="programlisting">int BZ2_bzDecompress ( bz_stream *strm );</pre>
<p>Provides more input and/out output buffer space for the
library. The caller maintains input and output buffers, and uses
<code class="computeroutput">BZ2_bzDecompress</code> to transfer
data between them.</p>
<p>Before each call to
<code class="computeroutput">BZ2_bzDecompress</code>,
<code class="computeroutput">next_in</code> should point at the
compressed data, and <code class="computeroutput">avail_in</code>
should indicate how many bytes the library may read.
<code class="computeroutput">BZ2_bzDecompress</code> updates
<code class="computeroutput">next_in</code>,
<code class="computeroutput">avail_in</code> and
<code class="computeroutput">total_in</code> to reflect the number
of bytes it has read.</p>
<p>Similarly, <code class="computeroutput">next_out</code> should
point to a buffer in which the uncompressed output is to be
placed, with <code class="computeroutput">avail_out</code>
indicating how much output space is available.
<code class="computeroutput">BZ2_bzCompress</code> updates
<code class="computeroutput">next_out</code>,
<code class="computeroutput">avail_out</code> and
<code class="computeroutput">total_out</code> to reflect the number
of bytes output.</p>
<p>You may provide and remove as little or as much data as you
like on each call of
<code class="computeroutput">BZ2_bzDecompress</code>. In the limit,
it is acceptable to supply and remove data one byte at a time,
although this would be terribly inefficient. You should always
ensure that at least one byte of output space is available at
each call.</p>
<p>Use of <code class="computeroutput">BZ2_bzDecompress</code> is
simpler than
<code class="computeroutput">BZ2_bzCompress</code>.</p>
<p>You should provide input and remove output as described
above, and repeatedly call
<code class="computeroutput">BZ2_bzDecompress</code> until
<code class="computeroutput">BZ_STREAM_END</code> is returned.
Appearance of <code class="computeroutput">BZ_STREAM_END</code>
denotes that <code class="computeroutput">BZ2_bzDecompress</code>
has detected the logical end of the compressed stream.
<code class="computeroutput">BZ2_bzDecompress</code> will not
produce <code class="computeroutput">BZ_STREAM_END</code> until all
output data has been placed into the output buffer, so once
<code class="computeroutput">BZ_STREAM_END</code> appears, you are
guaranteed to have available all the decompressed output, and
<code class="computeroutput">BZ2_bzDecompressEnd</code> can safely
be called.</p>
<p>If case of an error return value, you should call
<code class="computeroutput">BZ2_bzDecompressEnd</code> to clean up
and release memory.</p>
<p>Possible return values:</p>
<pre class="programlisting">BZ_PARAM_ERROR
if strm is NULL or strm->s is NULL
or strm->avail_out < 1
BZ_DATA_ERROR
if a data integrity error is detected in the compressed stream
BZ_DATA_ERROR_MAGIC
if the compressed stream doesn't begin with the right magic bytes
BZ_MEM_ERROR
if there wasn't enough memory available
BZ_STREAM_END
if the logical end of the data stream was detected and all
output in has been consumed, eg s-->avail_out > 0
BZ_OK
otherwise</pre>
<p>Allowable next actions:</p>
<pre class="programlisting">BZ2_bzDecompress
if BZ_OK was returned
BZ2_bzDecompressEnd
otherwise</pre>
</div>
<div class="sect2" title="3.3.6. BZ2_bzDecompressEnd">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzDecompress-end"></a>3.3.6. BZ2_bzDecompressEnd</h3></div></div></div>
<pre class="programlisting">int BZ2_bzDecompressEnd ( bz_stream *strm );</pre>
<p>Releases all memory associated with a decompression
stream.</p>
<p>Possible return values:</p>
<pre class="programlisting">BZ_PARAM_ERROR
if strm is NULL or strm->s is NULL
BZ_OK
otherwise</pre>
<p>Allowable next actions:</p>
<pre class="programlisting"> None.</pre>
</div>
</div>
<div class="sect1" title="3.4. High-level interface">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="hl-interface"></a>3.4. High-level interface</h2></div></div></div>
<p>This interface provides functions for reading and writing
<code class="computeroutput">bzip2</code> format files. First, some
general points.</p>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc"><p>All of the functions take an
<code class="computeroutput">int*</code> first argument,
<code class="computeroutput">bzerror</code>. After each call,
<code class="computeroutput">bzerror</code> should be consulted
first to determine the outcome of the call. If
<code class="computeroutput">bzerror</code> is
<code class="computeroutput">BZ_OK</code>, the call completed
successfully, and only then should the return value of the
function (if any) be consulted. If
<code class="computeroutput">bzerror</code> is
<code class="computeroutput">BZ_IO_ERROR</code>, there was an
error reading/writing the underlying compressed file, and you
should then consult <code class="computeroutput">errno</code> /
<code class="computeroutput">perror</code> to determine the cause
of the difficulty. <code class="computeroutput">bzerror</code>
may also be set to various other values; precise details are
given on a per-function basis below.</p></li>
<li class="listitem" style="list-style-type: disc"><p>If <code class="computeroutput">bzerror</code> indicates
an error (ie, anything except
<code class="computeroutput">BZ_OK</code> and
<code class="computeroutput">BZ_STREAM_END</code>), you should
immediately call
<code class="computeroutput">BZ2_bzReadClose</code> (or
<code class="computeroutput">BZ2_bzWriteClose</code>, depending on
whether you are attempting to read or to write) to free up all
resources associated with the stream. Once an error has been
indicated, behaviour of all calls except
<code class="computeroutput">BZ2_bzReadClose</code>
(<code class="computeroutput">BZ2_bzWriteClose</code>) is
undefined. The implication is that (1)
<code class="computeroutput">bzerror</code> should be checked
after each call, and (2) if
<code class="computeroutput">bzerror</code> indicates an error,
<code class="computeroutput">BZ2_bzReadClose</code>
(<code class="computeroutput">BZ2_bzWriteClose</code>) should then
be called to clean up.</p></li>
<li class="listitem" style="list-style-type: disc"><p>The <code class="computeroutput">FILE*</code> arguments
passed to <code class="computeroutput">BZ2_bzReadOpen</code> /
<code class="computeroutput">BZ2_bzWriteOpen</code> should be set
to binary mode. Most Unix systems will do this by default, but
other platforms, including Windows and Mac, will not. If you
omit this, you may encounter problems when moving code to new
platforms.</p></li>
<li class="listitem" style="list-style-type: disc"><p>Memory allocation requests are handled by
<code class="computeroutput">malloc</code> /
<code class="computeroutput">free</code>. At present there is no
facility for user-defined memory allocators in the file I/O
functions (could easily be added, though).</p></li>
</ul></div>
<div class="sect2" title="3.4.1. BZ2_bzReadOpen">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzreadopen"></a>3.4.1. BZ2_bzReadOpen</h3></div></div></div>
<pre class="programlisting">typedef void BZFILE;
BZFILE *BZ2_bzReadOpen( int *bzerror, FILE *f,
int verbosity, int small,
void *unused, int nUnused );</pre>
<p>Prepare to read compressed data from file handle
<code class="computeroutput">f</code>.
<code class="computeroutput">f</code> should refer to a file which
has been opened for reading, and for which the error indicator
(<code class="computeroutput">ferror(f)</code>)is not set. If
<code class="computeroutput">small</code> is 1, the library will try
to decompress using less memory, at the expense of speed.</p>
<p>For reasons explained below,
<code class="computeroutput">BZ2_bzRead</code> will decompress the
<code class="computeroutput">nUnused</code> bytes starting at
<code class="computeroutput">unused</code>, before starting to read
from the file <code class="computeroutput">f</code>. At most
<code class="computeroutput">BZ_MAX_UNUSED</code> bytes may be
supplied like this. If this facility is not required, you should
pass <code class="computeroutput">NULL</code> and
<code class="computeroutput">0</code> for
<code class="computeroutput">unused</code> and
n<code class="computeroutput">Unused</code> respectively.</p>
<p>For the meaning of parameters
<code class="computeroutput">small</code> and
<code class="computeroutput">verbosity</code>, see
<code class="computeroutput">BZ2_bzDecompressInit</code>.</p>
<p>The amount of memory needed to decompress a file cannot be
determined until the file's header has been read. So it is
possible that <code class="computeroutput">BZ2_bzReadOpen</code>
returns <code class="computeroutput">BZ_OK</code> but a subsequent
call of <code class="computeroutput">BZ2_bzRead</code> will return
<code class="computeroutput">BZ_MEM_ERROR</code>.</p>
<p>Possible assignments to
<code class="computeroutput">bzerror</code>:</p>
<pre class="programlisting">BZ_CONFIG_ERROR
if the library has been mis-compiled
BZ_PARAM_ERROR
if f is NULL
or small is neither 0 nor 1
or ( unused == NULL && nUnused != 0 )
or ( unused != NULL && !(0 <= nUnused <= BZ_MAX_UNUSED) )
BZ_IO_ERROR
if ferror(f) is nonzero
BZ_MEM_ERROR
if insufficient memory is available
BZ_OK
otherwise.</pre>
<p>Possible return values:</p>
<pre class="programlisting">Pointer to an abstract BZFILE
if bzerror is BZ_OK
NULL
otherwise</pre>
<p>Allowable next actions:</p>
<pre class="programlisting">BZ2_bzRead
if bzerror is BZ_OK
BZ2_bzClose
otherwise</pre>
</div>
<div class="sect2" title="3.4.2. BZ2_bzRead">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzread"></a>3.4.2. BZ2_bzRead</h3></div></div></div>
<pre class="programlisting">int BZ2_bzRead ( int *bzerror, BZFILE *b, void *buf, int len );</pre>
<p>Reads up to <code class="computeroutput">len</code>
(uncompressed) bytes from the compressed file
<code class="computeroutput">b</code> into the buffer
<code class="computeroutput">buf</code>. If the read was
successful, <code class="computeroutput">bzerror</code> is set to
<code class="computeroutput">BZ_OK</code> and the number of bytes
read is returned. If the logical end-of-stream was detected,
<code class="computeroutput">bzerror</code> will be set to
<code class="computeroutput">BZ_STREAM_END</code>, and the number of
bytes read is returned. All other
<code class="computeroutput">bzerror</code> values denote an
error.</p>
<p><code class="computeroutput">BZ2_bzRead</code> will supply
<code class="computeroutput">len</code> bytes, unless the logical
stream end is detected or an error occurs. Because of this, it
is possible to detect the stream end by observing when the number
of bytes returned is less than the number requested.
Nevertheless, this is regarded as inadvisable; you should instead
check <code class="computeroutput">bzerror</code> after every call
and watch out for
<code class="computeroutput">BZ_STREAM_END</code>.</p>
<p>Internally, <code class="computeroutput">BZ2_bzRead</code>
copies data from the compressed file in chunks of size
<code class="computeroutput">BZ_MAX_UNUSED</code> bytes before
decompressing it. If the file contains more bytes than strictly
needed to reach the logical end-of-stream,
<code class="computeroutput">BZ2_bzRead</code> will almost certainly
read some of the trailing data before signalling
<code class="computeroutput">BZ_SEQUENCE_END</code>. To collect the
read but unused data once
<code class="computeroutput">BZ_SEQUENCE_END</code> has appeared,
call <code class="computeroutput">BZ2_bzReadGetUnused</code>
immediately before
<code class="computeroutput">BZ2_bzReadClose</code>.</p>
<p>Possible assignments to
<code class="computeroutput">bzerror</code>:</p>
<pre class="programlisting">BZ_PARAM_ERROR
if b is NULL or buf is NULL or len < 0
BZ_SEQUENCE_ERROR
if b was opened with BZ2_bzWriteOpen
BZ_IO_ERROR
if there is an error reading from the compressed file
BZ_UNEXPECTED_EOF
if the compressed file ended before
the logical end-of-stream was detected
BZ_DATA_ERROR
if a data integrity error was detected in the compressed stream
BZ_DATA_ERROR_MAGIC
if the stream does not begin with the requisite header bytes
(ie, is not a bzip2 data file). This is really
a special case of BZ_DATA_ERROR.
BZ_MEM_ERROR
if insufficient memory was available
BZ_STREAM_END
if the logical end of stream was detected.
BZ_OK
otherwise.</pre>
<p>Possible return values:</p>
<pre class="programlisting">number of bytes read
if bzerror is BZ_OK or BZ_STREAM_END
undefined
otherwise</pre>
<p>Allowable next actions:</p>
<pre class="programlisting">collect data from buf, then BZ2_bzRead or BZ2_bzReadClose
if bzerror is BZ_OK
collect data from buf, then BZ2_bzReadClose or BZ2_bzReadGetUnused
if bzerror is BZ_SEQUENCE_END
BZ2_bzReadClose
otherwise</pre>
</div>
<div class="sect2" title="3.4.3. BZ2_bzReadGetUnused">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzreadgetunused"></a>3.4.3. BZ2_bzReadGetUnused</h3></div></div></div>
<pre class="programlisting">void BZ2_bzReadGetUnused( int* bzerror, BZFILE *b,
void** unused, int* nUnused );</pre>
<p>Returns data which was read from the compressed file but
was not needed to get to the logical end-of-stream.
<code class="computeroutput">*unused</code> is set to the address of
the data, and <code class="computeroutput">*nUnused</code> to the
number of bytes. <code class="computeroutput">*nUnused</code> will
be set to a value between <code class="computeroutput">0</code> and
<code class="computeroutput">BZ_MAX_UNUSED</code> inclusive.</p>
<p>This function may only be called once
<code class="computeroutput">BZ2_bzRead</code> has signalled
<code class="computeroutput">BZ_STREAM_END</code> but before
<code class="computeroutput">BZ2_bzReadClose</code>.</p>
<p>Possible assignments to
<code class="computeroutput">bzerror</code>:</p>
<pre class="programlisting">BZ_PARAM_ERROR
if b is NULL
or unused is NULL or nUnused is NULL
BZ_SEQUENCE_ERROR
if BZ_STREAM_END has not been signalled
or if b was opened with BZ2_bzWriteOpen
BZ_OK
otherwise</pre>
<p>Allowable next actions:</p>
<pre class="programlisting">BZ2_bzReadClose</pre>
</div>
<div class="sect2" title="3.4.4. BZ2_bzReadClose">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzreadclose"></a>3.4.4. BZ2_bzReadClose</h3></div></div></div>
<pre class="programlisting">void BZ2_bzReadClose ( int *bzerror, BZFILE *b );</pre>
<p>Releases all memory pertaining to the compressed file
<code class="computeroutput">b</code>.
<code class="computeroutput">BZ2_bzReadClose</code> does not call
<code class="computeroutput">fclose</code> on the underlying file
handle, so you should do that yourself if appropriate.
<code class="computeroutput">BZ2_bzReadClose</code> should be called
to clean up after all error situations.</p>
<p>Possible assignments to
<code class="computeroutput">bzerror</code>:</p>
<pre class="programlisting">BZ_SEQUENCE_ERROR
if b was opened with BZ2_bzOpenWrite
BZ_OK
otherwise</pre>
<p>Allowable next actions:</p>
<pre class="programlisting">none</pre>
</div>
<div class="sect2" title="3.4.5. BZ2_bzWriteOpen">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzwriteopen"></a>3.4.5. BZ2_bzWriteOpen</h3></div></div></div>
<pre class="programlisting">BZFILE *BZ2_bzWriteOpen( int *bzerror, FILE *f,
int blockSize100k, int verbosity,
int workFactor );</pre>
<p>Prepare to write compressed data to file handle
<code class="computeroutput">f</code>.
<code class="computeroutput">f</code> should refer to a file which
has been opened for writing, and for which the error indicator
(<code class="computeroutput">ferror(f)</code>)is not set.</p>
<p>For the meaning of parameters
<code class="computeroutput">blockSize100k</code>,
<code class="computeroutput">verbosity</code> and
<code class="computeroutput">workFactor</code>, see
<code class="computeroutput">BZ2_bzCompressInit</code>.</p>
<p>All required memory is allocated at this stage, so if the
call completes successfully,
<code class="computeroutput">BZ_MEM_ERROR</code> cannot be signalled
by a subsequent call to
<code class="computeroutput">BZ2_bzWrite</code>.</p>
<p>Possible assignments to
<code class="computeroutput">bzerror</code>:</p>
<pre class="programlisting">BZ_CONFIG_ERROR
if the library has been mis-compiled
BZ_PARAM_ERROR
if f is NULL
or blockSize100k < 1 or blockSize100k > 9
BZ_IO_ERROR
if ferror(f) is nonzero
BZ_MEM_ERROR
if insufficient memory is available
BZ_OK
otherwise</pre>
<p>Possible return values:</p>
<pre class="programlisting">Pointer to an abstract BZFILE
if bzerror is BZ_OK
NULL
otherwise</pre>
<p>Allowable next actions:</p>
<pre class="programlisting">BZ2_bzWrite
if bzerror is BZ_OK
(you could go directly to BZ2_bzWriteClose, but this would be pretty pointless)
BZ2_bzWriteClose
otherwise</pre>
</div>
<div class="sect2" title="3.4.6. BZ2_bzWrite">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzwrite"></a>3.4.6. BZ2_bzWrite</h3></div></div></div>
<pre class="programlisting">void BZ2_bzWrite ( int *bzerror, BZFILE *b, void *buf, int len );</pre>
<p>Absorbs <code class="computeroutput">len</code> bytes from the
buffer <code class="computeroutput">buf</code>, eventually to be
compressed and written to the file.</p>
<p>Possible assignments to
<code class="computeroutput">bzerror</code>:</p>
<pre class="programlisting">BZ_PARAM_ERROR
if b is NULL or buf is NULL or len < 0
BZ_SEQUENCE_ERROR
if b was opened with BZ2_bzReadOpen
BZ_IO_ERROR
if there is an error writing the compressed file.
BZ_OK
otherwise</pre>
</div>
<div class="sect2" title="3.4.7. BZ2_bzWriteClose">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzwriteclose"></a>3.4.7. BZ2_bzWriteClose</h3></div></div></div>
<pre class="programlisting">void BZ2_bzWriteClose( int *bzerror, BZFILE* f,
int abandon,
unsigned int* nbytes_in,
unsigned int* nbytes_out );
void BZ2_bzWriteClose64( int *bzerror, BZFILE* f,
int abandon,
unsigned int* nbytes_in_lo32,
unsigned int* nbytes_in_hi32,
unsigned int* nbytes_out_lo32,
unsigned int* nbytes_out_hi32 );</pre>
<p>Compresses and flushes to the compressed file all data so
far supplied by <code class="computeroutput">BZ2_bzWrite</code>.
The logical end-of-stream markers are also written, so subsequent
calls to <code class="computeroutput">BZ2_bzWrite</code> are
illegal. All memory associated with the compressed file
<code class="computeroutput">b</code> is released.
<code class="computeroutput">fflush</code> is called on the
compressed file, but it is not
<code class="computeroutput">fclose</code>'d.</p>
<p>If <code class="computeroutput">BZ2_bzWriteClose</code> is
called to clean up after an error, the only action is to release
the memory. The library records the error codes issued by
previous calls, so this situation will be detected automatically.
There is no attempt to complete the compression operation, nor to
<code class="computeroutput">fflush</code> the compressed file. You
can force this behaviour to happen even in the case of no error,
by passing a nonzero value to
<code class="computeroutput">abandon</code>.</p>
<p>If <code class="computeroutput">nbytes_in</code> is non-null,
<code class="computeroutput">*nbytes_in</code> will be set to be the
total volume of uncompressed data handled. Similarly,
<code class="computeroutput">nbytes_out</code> will be set to the
total volume of compressed data written. For compatibility with
older versions of the library,
<code class="computeroutput">BZ2_bzWriteClose</code> only yields the
lower 32 bits of these counts. Use
<code class="computeroutput">BZ2_bzWriteClose64</code> if you want
the full 64 bit counts. These two functions are otherwise
absolutely identical.</p>
<p>Possible assignments to
<code class="computeroutput">bzerror</code>:</p>
<pre class="programlisting">BZ_SEQUENCE_ERROR
if b was opened with BZ2_bzReadOpen
BZ_IO_ERROR
if there is an error writing the compressed file
BZ_OK
otherwise</pre>
</div>
<div class="sect2" title="3.4.8. Handling embedded compressed data streams">
<div class="titlepage"><div><div><h3 class="title">
<a name="embed"></a>3.4.8. Handling embedded compressed data streams</h3></div></div></div>
<p>The high-level library facilitates use of
<code class="computeroutput">bzip2</code> data streams which form
some part of a surrounding, larger data stream.</p>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc"><p>For writing, the library takes an open file handle,
writes compressed data to it,
<code class="computeroutput">fflush</code>es it but does not
<code class="computeroutput">fclose</code> it. The calling
application can write its own data before and after the
compressed data stream, using that same file handle.</p></li>
<li class="listitem" style="list-style-type: disc"><p>Reading is more complex, and the facilities are not as
general as they could be since generality is hard to reconcile
with efficiency. <code class="computeroutput">BZ2_bzRead</code>
reads from the compressed file in blocks of size
<code class="computeroutput">BZ_MAX_UNUSED</code> bytes, and in
doing so probably will overshoot the logical end of compressed
stream. To recover this data once decompression has ended,
call <code class="computeroutput">BZ2_bzReadGetUnused</code> after
the last call of <code class="computeroutput">BZ2_bzRead</code>
(the one returning
<code class="computeroutput">BZ_STREAM_END</code>) but before
calling
<code class="computeroutput">BZ2_bzReadClose</code>.</p></li>
</ul></div>
<p>This mechanism makes it easy to decompress multiple
<code class="computeroutput">bzip2</code> streams placed end-to-end.
As the end of one stream, when
<code class="computeroutput">BZ2_bzRead</code> returns
<code class="computeroutput">BZ_STREAM_END</code>, call
<code class="computeroutput">BZ2_bzReadGetUnused</code> to collect
the unused data (copy it into your own buffer somewhere). That
data forms the start of the next compressed stream. To start
uncompressing that next stream, call
<code class="computeroutput">BZ2_bzReadOpen</code> again, feeding in
the unused data via the <code class="computeroutput">unused</code> /
<code class="computeroutput">nUnused</code> parameters. Keep doing
this until <code class="computeroutput">BZ_STREAM_END</code> return
coincides with the physical end of file
(<code class="computeroutput">feof(f)</code>). In this situation
<code class="computeroutput">BZ2_bzReadGetUnused</code> will of
course return no data.</p>
<p>This should give some feel for how the high-level interface
can be used. If you require extra flexibility, you'll have to
bite the bullet and get to grips with the low-level
interface.</p>
</div>
<div class="sect2" title="3.4.9. Standard file-reading/writing code">
<div class="titlepage"><div><div><h3 class="title">
<a name="std-rdwr"></a>3.4.9. Standard file-reading/writing code</h3></div></div></div>
<p>Here's how you'd write data to a compressed file:</p>
<pre class="programlisting">FILE* f;
BZFILE* b;
int nBuf;
char buf[ /* whatever size you like */ ];
int bzerror;
int nWritten;
f = fopen ( "myfile.bz2", "w" );
if ( !f ) {
/* handle error */
}
b = BZ2_bzWriteOpen( &bzerror, f, 9 );
if (bzerror != BZ_OK) {
BZ2_bzWriteClose ( b );
/* handle error */
}
while ( /* condition */ ) {
/* get data to write into buf, and set nBuf appropriately */
nWritten = BZ2_bzWrite ( &bzerror, b, buf, nBuf );
if (bzerror == BZ_IO_ERROR) {
BZ2_bzWriteClose ( &bzerror, b );
/* handle error */
}
}
BZ2_bzWriteClose( &bzerror, b );
if (bzerror == BZ_IO_ERROR) {
/* handle error */
}</pre>
<p>And to read from a compressed file:</p>
<pre class="programlisting">FILE* f;
BZFILE* b;
int nBuf;
char buf[ /* whatever size you like */ ];
int bzerror;
int nWritten;
f = fopen ( "myfile.bz2", "r" );
if ( !f ) {
/* handle error */
}
b = BZ2_bzReadOpen ( &bzerror, f, 0, NULL, 0 );
if ( bzerror != BZ_OK ) {
BZ2_bzReadClose ( &bzerror, b );
/* handle error */
}
bzerror = BZ_OK;
while ( bzerror == BZ_OK && /* arbitrary other conditions */) {
nBuf = BZ2_bzRead ( &bzerror, b, buf, /* size of buf */ );
if ( bzerror == BZ_OK ) {
/* do something with buf[0 .. nBuf-1] */
}
}
if ( bzerror != BZ_STREAM_END ) {
BZ2_bzReadClose ( &bzerror, b );
/* handle error */
} else {
BZ2_bzReadClose ( &bzerror, b );
}</pre>
</div>
</div>
<div class="sect1" title="3.5. Utility functions">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="util-fns"></a>3.5. Utility functions</h2></div></div></div>
<div class="sect2" title="3.5.1. BZ2_bzBuffToBuffCompress">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzbufftobuffcompress"></a>3.5.1. BZ2_bzBuffToBuffCompress</h3></div></div></div>
<pre class="programlisting">int BZ2_bzBuffToBuffCompress( char* dest,
unsigned int* destLen,
char* source,
unsigned int sourceLen,
int blockSize100k,
int verbosity,
int workFactor );</pre>
<p>Attempts to compress the data in <code class="computeroutput">source[0
.. sourceLen-1]</code> into the destination buffer,
<code class="computeroutput">dest[0 .. *destLen-1]</code>. If the
destination buffer is big enough,
<code class="computeroutput">*destLen</code> is set to the size of
the compressed data, and <code class="computeroutput">BZ_OK</code>
is returned. If the compressed data won't fit,
<code class="computeroutput">*destLen</code> is unchanged, and
<code class="computeroutput">BZ_OUTBUFF_FULL</code> is
returned.</p>
<p>Compression in this manner is a one-shot event, done with a
single call to this function. The resulting compressed data is a
complete <code class="computeroutput">bzip2</code> format data
stream. There is no mechanism for making additional calls to
provide extra input data. If you want that kind of mechanism,
use the low-level interface.</p>
<p>For the meaning of parameters
<code class="computeroutput">blockSize100k</code>,
<code class="computeroutput">verbosity</code> and
<code class="computeroutput">workFactor</code>, see
<code class="computeroutput">BZ2_bzCompressInit</code>.</p>
<p>To guarantee that the compressed data will fit in its
buffer, allocate an output buffer of size 1% larger than the
uncompressed data, plus six hundred extra bytes.</p>
<p><code class="computeroutput">BZ2_bzBuffToBuffDecompress</code>
will not write data at or beyond
<code class="computeroutput">dest[*destLen]</code>, even in case of
buffer overflow.</p>
<p>Possible return values:</p>
<pre class="programlisting">BZ_CONFIG_ERROR
if the library has been mis-compiled
BZ_PARAM_ERROR
if dest is NULL or destLen is NULL
or blockSize100k < 1 or blockSize100k > 9
or verbosity < 0 or verbosity > 4
or workFactor < 0 or workFactor > 250
BZ_MEM_ERROR
if insufficient memory is available
BZ_OUTBUFF_FULL
if the size of the compressed data exceeds *destLen
BZ_OK
otherwise</pre>
</div>
<div class="sect2" title="3.5.2. BZ2_bzBuffToBuffDecompress">
<div class="titlepage"><div><div><h3 class="title">
<a name="bzbufftobuffdecompress"></a>3.5.2. BZ2_bzBuffToBuffDecompress</h3></div></div></div>
<pre class="programlisting">int BZ2_bzBuffToBuffDecompress( char* dest,
unsigned int* destLen,
char* source,
unsigned int sourceLen,
int small,
int verbosity );</pre>
<p>Attempts to decompress the data in <code class="computeroutput">source[0
.. sourceLen-1]</code> into the destination buffer,
<code class="computeroutput">dest[0 .. *destLen-1]</code>. If the
destination buffer is big enough,
<code class="computeroutput">*destLen</code> is set to the size of
the uncompressed data, and <code class="computeroutput">BZ_OK</code>
is returned. If the compressed data won't fit,
<code class="computeroutput">*destLen</code> is unchanged, and
<code class="computeroutput">BZ_OUTBUFF_FULL</code> is
returned.</p>
<p><code class="computeroutput">source</code> is assumed to hold
a complete <code class="computeroutput">bzip2</code> format data
stream.
<code class="computeroutput">BZ2_bzBuffToBuffDecompress</code> tries
to decompress the entirety of the stream into the output
buffer.</p>
<p>For the meaning of parameters
<code class="computeroutput">small</code> and
<code class="computeroutput">verbosity</code>, see
<code class="computeroutput">BZ2_bzDecompressInit</code>.</p>
<p>Because the compression ratio of the compressed data cannot
be known in advance, there is no easy way to guarantee that the
output buffer will be big enough. You may of course make
arrangements in your code to record the size of the uncompressed
data, but such a mechanism is beyond the scope of this
library.</p>
<p><code class="computeroutput">BZ2_bzBuffToBuffDecompress</code>
will not write data at or beyond
<code class="computeroutput">dest[*destLen]</code>, even in case of
buffer overflow.</p>
<p>Possible return values:</p>
<pre class="programlisting">BZ_CONFIG_ERROR
if the library has been mis-compiled
BZ_PARAM_ERROR
if dest is NULL or destLen is NULL
or small != 0 && small != 1
or verbosity < 0 or verbosity > 4
BZ_MEM_ERROR
if insufficient memory is available
BZ_OUTBUFF_FULL
if the size of the compressed data exceeds *destLen
BZ_DATA_ERROR
if a data integrity error was detected in the compressed data
BZ_DATA_ERROR_MAGIC
if the compressed data doesn't begin with the right magic bytes
BZ_UNEXPECTED_EOF
if the compressed data ends unexpectedly
BZ_OK
otherwise</pre>
</div>
</div>
<div class="sect1" title="3.6. zlib compatibility functions">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="zlib-compat"></a>3.6. zlib compatibility functions</h2></div></div></div>
<p>Yoshioka Tsuneo has contributed some functions to give
better <code class="computeroutput">zlib</code> compatibility.
These functions are <code class="computeroutput">BZ2_bzopen</code>,
<code class="computeroutput">BZ2_bzread</code>,
<code class="computeroutput">BZ2_bzwrite</code>,
<code class="computeroutput">BZ2_bzflush</code>,
<code class="computeroutput">BZ2_bzclose</code>,
<code class="computeroutput">BZ2_bzerror</code> and
<code class="computeroutput">BZ2_bzlibVersion</code>. These
functions are not (yet) officially part of the library. If they
break, you get to keep all the pieces. Nevertheless, I think
they work ok.</p>
<pre class="programlisting">typedef void BZFILE;
const char * BZ2_bzlibVersion ( void );</pre>
<p>Returns a string indicating the library version.</p>
<pre class="programlisting">BZFILE * BZ2_bzopen ( const char *path, const char *mode );
BZFILE * BZ2_bzdopen ( int fd, const char *mode );</pre>
<p>Opens a <code class="computeroutput">.bz2</code> file for
reading or writing, using either its name or a pre-existing file
descriptor. Analogous to <code class="computeroutput">fopen</code>
and <code class="computeroutput">fdopen</code>.</p>
<pre class="programlisting">int BZ2_bzread ( BZFILE* b, void* buf, int len );
int BZ2_bzwrite ( BZFILE* b, void* buf, int len );</pre>
<p>Reads/writes data from/to a previously opened
<code class="computeroutput">BZFILE</code>. Analogous to
<code class="computeroutput">fread</code> and
<code class="computeroutput">fwrite</code>.</p>
<pre class="programlisting">int BZ2_bzflush ( BZFILE* b );
void BZ2_bzclose ( BZFILE* b );</pre>
<p>Flushes/closes a <code class="computeroutput">BZFILE</code>.
<code class="computeroutput">BZ2_bzflush</code> doesn't actually do
anything. Analogous to <code class="computeroutput">fflush</code>
and <code class="computeroutput">fclose</code>.</p>
<pre class="programlisting">const char * BZ2_bzerror ( BZFILE *b, int *errnum )</pre>
<p>Returns a string describing the more recent error status of
<code class="computeroutput">b</code>, and also sets
<code class="computeroutput">*errnum</code> to its numerical
value.</p>
</div>
<div class="sect1" title="3.7. Using the library in a stdio-free environment">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="stdio-free"></a>3.7. Using the library in a stdio-free environment</h2></div></div></div>
<div class="sect2" title="3.7.1. Getting rid of stdio">
<div class="titlepage"><div><div><h3 class="title">
<a name="stdio-bye"></a>3.7.1. Getting rid of stdio</h3></div></div></div>
<p>In a deeply embedded application, you might want to use
just the memory-to-memory functions. You can do this
conveniently by compiling the library with preprocessor symbol
<code class="computeroutput">BZ_NO_STDIO</code> defined. Doing this
gives you a library containing only the following eight
functions:</p>
<p><code class="computeroutput">BZ2_bzCompressInit</code>,
<code class="computeroutput">BZ2_bzCompress</code>,
<code class="computeroutput">BZ2_bzCompressEnd</code>
<code class="computeroutput">BZ2_bzDecompressInit</code>,
<code class="computeroutput">BZ2_bzDecompress</code>,
<code class="computeroutput">BZ2_bzDecompressEnd</code>
<code class="computeroutput">BZ2_bzBuffToBuffCompress</code>,
<code class="computeroutput">BZ2_bzBuffToBuffDecompress</code></p>
<p>When compiled like this, all functions will ignore
<code class="computeroutput">verbosity</code> settings.</p>
</div>
<div class="sect2" title="3.7.2. Critical error handling">
<div class="titlepage"><div><div><h3 class="title">
<a name="critical-error"></a>3.7.2. Critical error handling</h3></div></div></div>
<p><code class="computeroutput">libbzip2</code> contains a number
of internal assertion checks which should, needless to say, never
be activated. Nevertheless, if an assertion should fail,
behaviour depends on whether or not the library was compiled with
<code class="computeroutput">BZ_NO_STDIO</code> set.</p>
<p>For a normal compile, an assertion failure yields the
message:</p>
<div class="blockquote"><blockquote class="blockquote">
<p>bzip2/libbzip2: internal error number N.</p>
<p>This is a bug in bzip2/libbzip2, 1.0.6 of 6 September 2010.
Please report it to me at: jseward@bzip.org. If this happened
when you were using some program which uses libbzip2 as a
component, you should also report this bug to the author(s)
of that program. Please make an effort to report this bug;
timely and accurate bug reports eventually lead to higher
quality software. Thanks. Julian Seward, 6 September 2010.
</p>
</blockquote></div>
<p>where <code class="computeroutput">N</code> is some error code
number. If <code class="computeroutput">N == 1007</code>, it also
prints some extra text advising the reader that unreliable memory
is often associated with internal error 1007. (This is a
frequently-observed-phenomenon with versions 1.0.0/1.0.1).</p>
<p><code class="computeroutput">exit(3)</code> is then
called.</p>
<p>For a <code class="computeroutput">stdio</code>-free library,
assertion failures result in a call to a function declared
as:</p>
<pre class="programlisting">extern void bz_internal_error ( int errcode );</pre>
<p>The relevant code is passed as a parameter. You should
supply such a function.</p>
<p>In either case, once an assertion failure has occurred, any
<code class="computeroutput">bz_stream</code> records involved can
be regarded as invalid. You should not attempt to resume normal
operation with them.</p>
<p>You may, of course, change critical error handling to suit
your needs. As I said above, critical errors indicate bugs in
the library and should not occur. All "normal" error situations
are indicated via error return codes from functions, and can be
recovered from.</p>
</div>
</div>
<div class="sect1" title="3.8. Making a Windows DLL">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="win-dll"></a>3.8. Making a Windows DLL</h2></div></div></div>
<p>Everything related to Windows has been contributed by
Yoshioka Tsuneo
(<code class="computeroutput">tsuneo@rr.iij4u.or.jp</code>), so
you should send your queries to him (but perhaps Cc: me,
<code class="computeroutput">jseward@bzip.org</code>).</p>
<p>My vague understanding of what to do is: using Visual C++
5.0, open the project file
<code class="computeroutput">libbz2.dsp</code>, and build. That's
all.</p>
<p>If you can't open the project file for some reason, make a
new one, naming these files:
<code class="computeroutput">blocksort.c</code>,
<code class="computeroutput">bzlib.c</code>,
<code class="computeroutput">compress.c</code>,
<code class="computeroutput">crctable.c</code>,
<code class="computeroutput">decompress.c</code>,
<code class="computeroutput">huffman.c</code>,
<code class="computeroutput">randtable.c</code> and
<code class="computeroutput">libbz2.def</code>. You will also need
to name the header files <code class="computeroutput">bzlib.h</code>
and <code class="computeroutput">bzlib_private.h</code>.</p>
<p>If you don't use VC++, you may need to define the
proprocessor symbol
<code class="computeroutput">_WIN32</code>.</p>
<p>Finally, <code class="computeroutput">dlltest.c</code> is a
sample program using the DLL. It has a project file,
<code class="computeroutput">dlltest.dsp</code>.</p>
<p>If you just want a makefile for Visual C, have a look at
<code class="computeroutput">makefile.msc</code>.</p>
<p>Be aware that if you compile
<code class="computeroutput">bzip2</code> itself on Win32, you must
set <code class="computeroutput">BZ_UNIX</code> to 0 and
<code class="computeroutput">BZ_LCCWIN32</code> to 1, in the file
<code class="computeroutput">bzip2.c</code>, before compiling.
Otherwise the resulting binary won't work correctly.</p>
<p>I haven't tried any of this stuff myself, but it all looks
plausible.</p>
</div>
</div>
<div class="chapter" title="4. Miscellanea">
<div class="titlepage"><div><div><h2 class="title">
<a name="misc"></a>4. Miscellanea</h2></div></div></div>
<div class="toc">
<p><b>Table of Contents</b></p>
<dl>
<dt><span class="sect1"><a href="#limits">4.1. Limitations of the compressed file format</a></span></dt>
<dt><span class="sect1"><a href="#port-issues">4.2. Portability issues</a></span></dt>
<dt><span class="sect1"><a href="#bugs">4.3. Reporting bugs</a></span></dt>
<dt><span class="sect1"><a href="#package">4.4. Did you get the right package?</a></span></dt>
<dt><span class="sect1"><a href="#reading">4.5. Further Reading</a></span></dt>
</dl>
</div>
<p>These are just some random thoughts of mine. Your mileage
may vary.</p>
<div class="sect1" title="4.1. Limitations of the compressed file format">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="limits"></a>4.1. Limitations of the compressed file format</h2></div></div></div>
<p><code class="computeroutput">bzip2-1.0.X</code>,
<code class="computeroutput">0.9.5</code> and
<code class="computeroutput">0.9.0</code> use exactly the same file
format as the original version,
<code class="computeroutput">bzip2-0.1</code>. This decision was
made in the interests of stability. Creating yet another
incompatible compressed file format would create further
confusion and disruption for users.</p>
<p>Nevertheless, this is not a painless decision. Development
work since the release of
<code class="computeroutput">bzip2-0.1</code> in August 1997 has
shown complexities in the file format which slow down
decompression and, in retrospect, are unnecessary. These
are:</p>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc"><p>The run-length encoder, which is the first of the
compression transformations, is entirely irrelevant. The
original purpose was to protect the sorting algorithm from the
very worst case input: a string of repeated symbols. But
algorithm steps Q6a and Q6b in the original Burrows-Wheeler
technical report (SRC-124) show how repeats can be handled
without difficulty in block sorting.</p></li>
<li class="listitem" style="list-style-type: disc">
<p>The randomisation mechanism doesn't really need to be
there. Udi Manber and Gene Myers published a suffix array
construction algorithm a few years back, which can be employed
to sort any block, no matter how repetitive, in O(N log N)
time. Subsequent work by Kunihiko Sadakane has produced a
derivative O(N (log N)^2) algorithm which usually outperforms
the Manber-Myers algorithm.</p>
<p>I could have changed to Sadakane's algorithm, but I find
it to be slower than <code class="computeroutput">bzip2</code>'s
existing algorithm for most inputs, and the randomisation
mechanism protects adequately against bad cases. I didn't
think it was a good tradeoff to make. Partly this is due to
the fact that I was not flooded with email complaints about
<code class="computeroutput">bzip2-0.1</code>'s performance on
repetitive data, so perhaps it isn't a problem for real
inputs.</p>
<p>Probably the best long-term solution, and the one I have
incorporated into 0.9.5 and above, is to use the existing
sorting algorithm initially, and fall back to a O(N (log N)^2)
algorithm if the standard algorithm gets into
difficulties.</p>
</li>
<li class="listitem" style="list-style-type: disc"><p>The compressed file format was never designed to be
handled by a library, and I have had to jump though some hoops
to produce an efficient implementation of decompression. It's
a bit hairy. Try passing
<code class="computeroutput">decompress.c</code> through the C
preprocessor and you'll see what I mean. Much of this
complexity could have been avoided if the compressed size of
each block of data was recorded in the data stream.</p></li>
<li class="listitem" style="list-style-type: disc"><p>An Adler-32 checksum, rather than a CRC32 checksum,
would be faster to compute.</p></li>
</ul></div>
<p>It would be fair to say that the
<code class="computeroutput">bzip2</code> format was frozen before I
properly and fully understood the performance consequences of
doing so.</p>
<p>Improvements which I was able to incorporate into 0.9.0,
despite using the same file format, are:</p>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc"><p>Single array implementation of the inverse BWT. This
significantly speeds up decompression, presumably because it
reduces the number of cache misses.</p></li>
<li class="listitem" style="list-style-type: disc"><p>Faster inverse MTF transform for large MTF values.
The new implementation is based on the notion of sliding blocks
of values.</p></li>
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2-0.9.0</code> now reads
and writes files with <code class="computeroutput">fread</code>
and <code class="computeroutput">fwrite</code>; version 0.1 used
<code class="computeroutput">putc</code> and
<code class="computeroutput">getc</code>. Duh! Well, you live
and learn.</p></li>
</ul></div>
<p>Further ahead, it would be nice to be able to do random
access into files. This will require some careful design of
compressed file formats.</p>
</div>
<div class="sect1" title="4.2. Portability issues">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="port-issues"></a>4.2. Portability issues</h2></div></div></div>
<p>After some consideration, I have decided not to use GNU
<code class="computeroutput">autoconf</code> to configure 0.9.5 or
1.0.</p>
<p><code class="computeroutput">autoconf</code>, admirable and
wonderful though it is, mainly assists with portability problems
between Unix-like platforms. But
<code class="computeroutput">bzip2</code> doesn't have much in the
way of portability problems on Unix; most of the difficulties
appear when porting to the Mac, or to Microsoft's operating
systems. <code class="computeroutput">autoconf</code> doesn't help
in those cases, and brings in a whole load of new
complexity.</p>
<p>Most people should be able to compile the library and
program under Unix straight out-of-the-box, so to speak,
especially if you have a version of GNU C available.</p>
<p>There are a couple of
<code class="computeroutput">__inline__</code> directives in the
code. GNU C (<code class="computeroutput">gcc</code>) should be
able to handle them. If you're not using GNU C, your C compiler
shouldn't see them at all. If your compiler does, for some
reason, see them and doesn't like them, just
<code class="computeroutput">#define</code>
<code class="computeroutput">__inline__</code> to be
<code class="computeroutput">/* */</code>. One easy way to do this
is to compile with the flag
<code class="computeroutput">-D__inline__=</code>, which should be
understood by most Unix compilers.</p>
<p>If you still have difficulties, try compiling with the
macro <code class="computeroutput">BZ_STRICT_ANSI</code> defined.
This should enable you to build the library in a strictly ANSI
compliant environment. Building the program itself like this is
dangerous and not supported, since you remove
<code class="computeroutput">bzip2</code>'s checks against
compressing directories, symbolic links, devices, and other
not-really-a-file entities. This could cause filesystem
corruption!</p>
<p>One other thing: if you create a
<code class="computeroutput">bzip2</code> binary for public distribution,
please consider linking it statically (<code class="computeroutput">gcc
-static</code>). This avoids all sorts of library-version
issues that others may encounter later on.</p>
<p>If you build <code class="computeroutput">bzip2</code> on
Win32, you must set <code class="computeroutput">BZ_UNIX</code> to 0
and <code class="computeroutput">BZ_LCCWIN32</code> to 1, in the
file <code class="computeroutput">bzip2.c</code>, before compiling.
Otherwise the resulting binary won't work correctly.</p>
</div>
<div class="sect1" title="4.3. Reporting bugs">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="bugs"></a>4.3. Reporting bugs</h2></div></div></div>
<p>I tried pretty hard to make sure
<code class="computeroutput">bzip2</code> is bug free, both by
design and by testing. Hopefully you'll never need to read this
section for real.</p>
<p>Nevertheless, if <code class="computeroutput">bzip2</code> dies
with a segmentation fault, a bus error or an internal assertion
failure, it will ask you to email me a bug report. Experience from
years of feedback of bzip2 users indicates that almost all these
problems can be traced to either compiler bugs or hardware
problems.</p>
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
<li class="listitem" style="list-style-type: disc">
<p>Recompile the program with no optimisation, and
see if it works. And/or try a different compiler. I heard all
sorts of stories about various flavours of GNU C (and other
compilers) generating bad code for
<code class="computeroutput">bzip2</code>, and I've run across two
such examples myself.</p>
<p>2.7.X versions of GNU C are known to generate bad code
from time to time, at high optimisation levels. If you get
problems, try using the flags
<code class="computeroutput">-O2</code>
<code class="computeroutput">-fomit-frame-pointer</code>
<code class="computeroutput">-fno-strength-reduce</code>. You
should specifically <span class="emphasis"><em>not</em></span> use
<code class="computeroutput">-funroll-loops</code>.</p>
<p>You may notice that the Makefile runs six tests as part
of the build process. If the program passes all of these, it's
a pretty good (but not 100%) indication that the compiler has
done its job correctly.</p>
</li>
<li class="listitem" style="list-style-type: disc">
<p>If <code class="computeroutput">bzip2</code>
crashes randomly, and the crashes are not repeatable, you may
have a flaky memory subsystem.
<code class="computeroutput">bzip2</code> really hammers your
memory hierarchy, and if it's a bit marginal, you may get these
problems. Ditto if your disk or I/O subsystem is slowly
failing. Yup, this really does happen.</p>
<p>Try using a different machine of the same type, and see
if you can repeat the problem.</p>
</li>
<li class="listitem" style="list-style-type: disc"><p>This isn't really a bug, but ... If
<code class="computeroutput">bzip2</code> tells you your file is
corrupted on decompression, and you obtained the file via FTP,
there is a possibility that you forgot to tell FTP to do a
binary mode transfer. That absolutely will cause the file to
be non-decompressible. You'll have to transfer it
again.</p></li>
</ul></div>
<p>If you've incorporated
<code class="computeroutput">libbzip2</code> into your own program
and are getting problems, please, please, please, check that the
parameters you are passing in calls to the library, are correct,
and in accordance with what the documentation says is allowable.
I have tried to make the library robust against such problems,
but I'm sure I haven't succeeded.</p>
<p>Finally, if the above comments don't help, you'll have to
send me a bug report. Now, it's just amazing how many people
will send me a bug report saying something like:</p>
<pre class="programlisting">bzip2 crashed with segmentation fault on my machine</pre>
<p>and absolutely nothing else. Needless to say, a such a
report is <span class="emphasis"><em>totally, utterly, completely and
comprehensively 100% useless; a waste of your time, my time, and
net bandwidth</em></span>. With no details at all, there's no way
I can possibly begin to figure out what the problem is.</p>
<p>The rules of the game are: facts, facts, facts. Don't omit
them because "oh, they won't be relevant". At the bare
minimum:</p>
<pre class="programlisting">Machine type. Operating system version.
Exact version of bzip2 (do bzip2 -V).
Exact version of the compiler used.
Flags passed to the compiler.</pre>
<p>However, the most important single thing that will help me
is the file that you were trying to compress or decompress at the
time the problem happened. Without that, my ability to do
anything more than speculate about the cause, is limited.</p>
</div>
<div class="sect1" title="4.4. Did you get the right package?">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="package"></a>4.4. Did you get the right package?</h2></div></div></div>
<p><code class="computeroutput">bzip2</code> is a resource hog.
It soaks up large amounts of CPU cycles and memory. Also, it
gives very large latencies. In the worst case, you can feed many
megabytes of uncompressed data into the library before getting
any compressed output, so this probably rules out applications
requiring interactive behaviour.</p>
<p>These aren't faults of my implementation, I hope, but more
an intrinsic property of the Burrows-Wheeler transform
(unfortunately). Maybe this isn't what you want.</p>
<p>If you want a compressor and/or library which is faster,
uses less memory but gets pretty good compression, and has
minimal latency, consider Jean-loup Gailly's and Mark Adler's
work, <code class="computeroutput">zlib-1.2.1</code> and
<code class="computeroutput">gzip-1.2.4</code>. Look for them at
<a class="ulink" href="http://www.zlib.org" target="_top">http://www.zlib.org</a> and
<a class="ulink" href="http://www.gzip.org" target="_top">http://www.gzip.org</a>
respectively.</p>
<p>For something faster and lighter still, you might try Markus F
X J Oberhumer's <code class="computeroutput">LZO</code> real-time
compression/decompression library, at
<a class="ulink" href="http://www.oberhumer.com/opensource" target="_top">http://www.oberhumer.com/opensource</a>.</p>
</div>
<div class="sect1" title="4.5. Further Reading">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="reading"></a>4.5. Further Reading</h2></div></div></div>
<p><code class="computeroutput">bzip2</code> is not research
work, in the sense that it doesn't present any new ideas.
Rather, it's an engineering exercise based on existing
ideas.</p>
<p>Four documents describe essentially all the ideas behind
<code class="computeroutput">bzip2</code>:</p>
<div class="literallayout"><p>Michael Burrows and D. J. Wheeler:<br>
"A block-sorting lossless data compression algorithm"<br>
10th May 1994. <br>
Digital SRC Research Report 124.<br>
ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.gz<br>
If you have trouble finding it, try searching at the<br>
New Zealand Digital Library, http://www.nzdl.org.<br>
<br>
Daniel S. Hirschberg and Debra A. LeLewer<br>
"Efficient Decoding of Prefix Codes"<br>
Communications of the ACM, April 1990, Vol 33, Number 4.<br>
You might be able to get an electronic copy of this<br>
from the ACM Digital Library.<br>
<br>
David J. Wheeler<br>
Program bred3.c and accompanying document bred3.ps.<br>
This contains the idea behind the multi-table Huffman coding scheme.<br>
ftp://ftp.cl.cam.ac.uk/users/djw3/<br>
<br>
Jon L. Bentley and Robert Sedgewick<br>
"Fast Algorithms for Sorting and Searching Strings"<br>
Available from Sedgewick's web page,<br>
www.cs.princeton.edu/~rs<br>
</p></div>
<p>The following paper gives valuable additional insights into
the algorithm, but is not immediately the basis of any code used
in bzip2.</p>
<div class="literallayout"><p>Peter Fenwick:<br>
Block Sorting Text Compression<br>
Proceedings of the 19th Australasian Computer Science Conference,<br>
Melbourne, Australia. Jan 31 - Feb 2, 1996.<br>
ftp://ftp.cs.auckland.ac.nz/pub/peter-f/ACSC96paper.ps</p></div>
<p>Kunihiko Sadakane's sorting algorithm, mentioned above, is
available from:</p>
<div class="literallayout"><p>http://naomi.is.s.u-tokyo.ac.jp/~sada/papers/Sada98b.ps.gz<br>
</p></div>
<p>The Manber-Myers suffix array construction algorithm is
described in a paper available from:</p>
<div class="literallayout"><p>http://www.cs.arizona.edu/people/gene/PAPERS/suffix.ps<br>
</p></div>
<p>Finally, the following papers document some
investigations I made into the performance of sorting
and decompression algorithms:</p>
<div class="literallayout"><p>Julian Seward<br>
On the Performance of BWT Sorting Algorithms<br>
Proceedings of the IEEE Data Compression Conference 2000<br>
Snowbird, Utah. 28-30 March 2000.<br>
<br>
Julian Seward<br>
Space-time Tradeoffs in the Inverse B-W Transform<br>
Proceedings of the IEEE Data Compression Conference 2001<br>
Snowbird, Utah. 27-29 March 2001.<br>
</p></div>
</div>
</div>
</div></body>
</html>
|