From 5df79c53745fde5d6c3340a2979b1429cd5892c1 Mon Sep 17 00:00:00 2001 From: Henrik Rydberg Date: Sat, 8 Oct 2011 20:30:28 +0200 Subject: Initial import of htcd system 1.0 Signed-off-by: Henrik Rydberg --- src/hdb/README | 154 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 154 insertions(+) create mode 100644 src/hdb/README (limited to 'src/hdb/README') diff --git a/src/hdb/README b/src/hdb/README new file mode 100644 index 0000000..0db7269 --- /dev/null +++ b/src/hdb/README @@ -0,0 +1,154 @@ +/************************************************************************* + * + * HTCd - Copyright (C) 1998-2006 Henrik Rydberg + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + */ + +HDB/1.2 FILE FORMAT SPECIFICATION Sat Feb 24 18:38:25 MET 2001 + +MT AND MP SAFE + +One single format utilizing memory maps, and which is designed to +work safely between processes and within a multi-threading process. +To accomplish this, there are two types of locking mechanisms - +one inter-process lock in the file header, and one MT lock implemented +the usual way. + +The format is robust within each thread, except for changes in the +table structure, which generates fatal error_t errors. + +The format includes a finite (32) number of tables, which should not +be a real problem. Besides, a possible change is to move the frame +pointer to a position inside, so that additional header could be +added if needed. + +FILEFORMAT in HDB/1.2 + +BYTES 1.2 NAME DESCRIPTION +32 IDENT Identification header +4 LOCK Inter-process lock +4 MINOR Minor modification state number +4 MAJOR Major modification state number +4 FATAL Fatal modification state number +4 FILESIZE Size of file +4 FILETOP Size of used part +4 TABLES Number of table name strings +4 INDEX Which column is the order column. +32 COMPNAME Comparison name +32*32 TABLENAMES Table names +4 FRAME Start of record data +* RECORDS Record list + 4 mem amount of memory for this record + 4 caps the capacity of this record - 0 means deleted + mem-8 data The data follows +0 ZEROS The rest of the file is padded with zeros. + +The records are always staying where they land; a minor change means +a change in-place, a major change means a change leading to an index +update, which could be a deletion, an insertion. Those are updated at +will, and not seen until wanted. A fatal change is a change in the +table structure, i.e., an insertion or deletion of a table. Also a +table name change counts as fatal. The filesize used to be (earlier +versions) a state needing attention as well, but not anymore. + +-- + +HDB/1.11 FILE FORMAT SPECIFICATION Mon Jan 29 03:51:41 MET 2001 + +STATIC/DYNAMIC mapping + +The header field determines whether the mapping is dynamic or static, +i.e., static mapping creates a shared mmap and edits the records +directly, whereas the dynamic mapping creates a private map and +stores the changes on save only. + +The format changed quite a lot after all... + +FILEFORMAT in HDB/1.11 + +BYTES 1.11 NAME DESCRIPTION +32 IDENT Identification header +4 MODNUM Modification number +4 LOCK Soft lock +4 FILESIZE Size of file +4 FILETOP Size of used part +4 FIXED Whether fixed mapping or not +4 TABLES Number of table name strings +4 INDEX Which column is the order column. +32 COMPNAME Comparison name +4 RECORDS Number of records in file. + tables + 4,* name table name + 4,* defaults default string + 4 caps default sizes + [4],* records TABLES[RECORDS] + +Size info [4] always refers to the stringref-size, but the number of +bytes actually written is as follows: for fixed maps, it is tabs[].caps, +and for dynamic maps, it is capsize(), which is defined as (n=stringref-size) +(4+n)&~3. + +-- + +HDB/1.10 FILE FORMAT SPECIFICATION Thu Jan 25 00:42:49 MET 2001 + +The HDB/1.0 format has been used widely in the software +for our group (MST) during the last three years of so. +The HDB/1.1 file format is binary compatible with 1.0, +and merely consists of a formalization, along with a +change in the internal manipulation scheme and a new +hdb editor. + +FILE FORMAT IN 1.0 and 1.1 + +BYTES 1.0/1.1 NAME DESCRIPTION +32/variable HEAD Identification header. + Used to be exactly 32 bytes, + but see no point in that. +4 TABLES Number of table name strings + 4 dim bytes in string + * data string data +4 DEFAULTS Number of default strings + 4 dim bytes in string + * data string data +4 ORDERBY Which column is the order column. +4 dim Number of bytes in compname. +* data string data +4 RECORDS Number of records in file. + 4 tables + 4 dim + * data +0/variable FOOTER File ends. Simple Footer. + +ADDITIONAL CHANGES BETWEEN 1.0 and 1.1 + +The major difference - and the rationale for 1.1 - is +the possiblity to edit files directly, as in mmap. +To be able to do this, there is a need to force a minimum +size to each field. This is done by padding the data +with trailing zeros. This is ok since only string data +is stored in the fields, and never binary data. Just think +of it; if it was binary data, it would not be replaced +by something the same size anyways, so thats forbidden +in the direct-editing mode. It makes no difference to +the normal operation. + +The default sizes for new fields are taken from the defaults +table, which again is completely compatible. The only +difference is that the zeros are stored as well, and hence +cannot be defined just by the default string, but need an editor +that specifies the sizes. + -- cgit v1.2.3