{
"subject": "Re: RFC: ship block chain 1-74000 with release tarballs?",
"content": {
"format": "html",
"body": "<div class=\"post\">It seems like you're inclined to assume everything is wrong more than is actually so.<br/><br/>Writing the block index is light work. Building the tx index is much more random access per block. I suspect reading all the prev txins is what's slow. Read caching would help that. It's best if the DB does that. Maybe it has a setting for how much cache memory to use.<br/><br/><div class=\"quoteheader\">Quote</div><div class=\"quote\">1) bitcoin should be opening databases, not just environment, at program startup, and closing database at program shutdown. <br/></div>Already does that. See CDB. The lifetime of the (for instance) CTxDB object is only to support database transactions and to know if anything is still using the database at shutdown.<br/><br/><div class=\"quoteheader\">Quote</div><div class=\"quote\">And, additionally, bitcoin forces a database checkpoint, pushing all transactions from log into main database.<br/></div>If it was doing that it would be much slower. It's supposed to be only once a minute or 500 blocks:<br/><br/> if (strFile == \"blkindex.dat\" && IsInitialBlockDownload() && nBestHeight % 500 != 0)<br/> nMinutes = 1;<br/> dbenv.txn_checkpoint(0, nMinutes, 0);<br/><br/>Probably should add this:<br/> if (!fReadOnly)<br/> dbenv.txn_checkpoint(0, nMinutes, 0);<br/><br/><div class=\"quoteheader\">Quote</div><div class=\"quote\">2) For the initial block download, txn commit should occur once every N records, not every record. I suggest N=1000.<br/></div>Does transaction commit imply flush? That seems surprising to me. I assume a database op wrapped in a transaction would be logged like any other database op. Many database applications need to wrap almost every pair of ops in a transaction, such as moving money from one account to another. (debit a, credit b) I can't imagine they're required to batch all their stuff up themselves.<br/><br/>In the following cases, would case 1 flush once and case 2 flush twice?<br/><br/>case 1:<br/>write<br/>write<br/>write<br/>write<br/>checkpoint<br/><br/>case 2:<br/>begin transaction<br/>write<br/>write<br/>commit transaction<br/>begin transaction<br/>write<br/>write<br/>commit transaction<br/>checkpoint<br/><br/>Contorting our database usage will not be the right approach. It's going to be BDB settings and caching.</div>"
},
"source": {
"name": "Bitcoin Forum",
"url": "https://bitcointalk.org/index.php?topic=1931.msg25449#msg25449"
},
"date": "2010-11-29T20:19:12Z"
}