/************************************************************************* * * HTCd - Copyright (C) 1998-2006 Henrik Rydberg * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */ HDB/1.2 FILE FORMAT SPECIFICATION Sat Feb 24 18:38:25 MET 2001 MT AND MP SAFE One single format utilizing memory maps, and which is designed to work safely between processes and within a multi-threading process. To accomplish this, there are two types of locking mechanisms - one inter-process lock in the file header, and one MT lock implemented the usual way. The format is robust within each thread, except for changes in the table structure, which generates fatal error_t errors. The format includes a finite (32) number of tables, which should not be a real problem. Besides, a possible change is to move the frame pointer to a position inside, so that additional header could be added if needed. FILEFORMAT in HDB/1.2 BYTES 1.2 NAME DESCRIPTION 32 IDENT Identification header 4 LOCK Inter-process lock 4 MINOR Minor modification state number 4 MAJOR Major modification state number 4 FATAL Fatal modification state number 4 FILESIZE Size of file 4 FILETOP Size of used part 4 TABLES Number of table name strings 4 INDEX Which column is the order column. 32 COMPNAME Comparison name 32*32 TABLENAMES Table names 4 FRAME Start of record data * RECORDS Record list 4 mem amount of memory for this record 4 caps the capacity of this record - 0 means deleted mem-8 data The data follows 0 ZEROS The rest of the file is padded with zeros. The records are always staying where they land; a minor change means a change in-place, a major change means a change leading to an index update, which could be a deletion, an insertion. Those are updated at will, and not seen until wanted. A fatal change is a change in the table structure, i.e., an insertion or deletion of a table. Also a table name change counts as fatal. The filesize used to be (earlier versions) a state needing attention as well, but not anymore. -- HDB/1.11 FILE FORMAT SPECIFICATION Mon Jan 29 03:51:41 MET 2001 STATIC/DYNAMIC mapping The header field determines whether the mapping is dynamic or static, i.e., static mapping creates a shared mmap and edits the records directly, whereas the dynamic mapping creates a private map and stores the changes on save only. The format changed quite a lot after all... FILEFORMAT in HDB/1.11 BYTES 1.11 NAME DESCRIPTION 32 IDENT Identification header 4 MODNUM Modification number 4 LOCK Soft lock 4 FILESIZE Size of file 4 FILETOP Size of used part 4 FIXED Whether fixed mapping or not 4 TABLES Number of table name strings 4 INDEX Which column is the order column. 32 COMPNAME Comparison name 4 RECORDS Number of records in file. tables 4,* name table name 4,* defaults default string 4 caps default sizes [4],* records TABLES[RECORDS] Size info [4] always refers to the stringref-size, but the number of bytes actually written is as follows: for fixed maps, it is tabs[].caps, and for dynamic maps, it is capsize(), which is defined as (n=stringref-size) (4+n)&~3. -- HDB/1.10 FILE FORMAT SPECIFICATION Thu Jan 25 00:42:49 MET 2001 The HDB/1.0 format has been used widely in the software for our group (MST) during the last three years of so. The HDB/1.1 file format is binary compatible with 1.0, and merely consists of a formalization, along with a change in the internal manipulation scheme and a new hdb editor. FILE FORMAT IN 1.0 and 1.1 BYTES 1.0/1.1 NAME DESCRIPTION 32/variable HEAD Identification header. Used to be exactly 32 bytes, but see no point in that. 4 TABLES Number of table name strings 4 dim bytes in string * data string data 4 DEFAULTS Number of default strings 4 dim bytes in string * data string data 4 ORDERBY Which column is the order column. 4 dim Number of bytes in compname. * data string data 4 RECORDS Number of records in file. 4 tables 4 dim * data 0/variable FOOTER File ends. Simple Footer. ADDITIONAL CHANGES BETWEEN 1.0 and 1.1 The major difference - and the rationale for 1.1 - is the possiblity to edit files directly, as in mmap. To be able to do this, there is a need to force a minimum size to each field. This is done by padding the data with trailing zeros. This is ok since only string data is stored in the fields, and never binary data. Just think of it; if it was binary data, it would not be replaced by something the same size anyways, so thats forbidden in the direct-editing mode. It makes no difference to the normal operation. The default sizes for new fields are taken from the defaults table, which again is completely compatible. The only difference is that the zeros are stored as well, and hence cannot be defined just by the default string, but need an editor that specifies the sizes.