Skip to content

Large files

Gedcom::read() holds the whole tree in memory, packed: a 50 MB file takes about 430 MB. For files that do not fit, or when one pass is enough, there are three ways that keep memory flat.

Memory Access Editing
Gedcom::stream() One record Records in file order, once A record at a time, written out as you go
Gedcom::open() ~100 bytes per record Any record, by xref Read only; copy() of a record, load() of all
Gedcom::convertFile() One record — —

Streaming

Each record is a document of its own, with the lines and diagnostics read() would give it. The diagnostics of the whole file (encoding, HEAD, TRLR, version) are known after the last one.

use Korin\Gedcom\Gedcom;

$stream = Gedcom::stream('huge.ged');       // a path or an open stream resource
echo $stream->version()->value, ' ', $stream->encoding()->value, PHP_EOL;

$out = fopen('huge-edited.ged', 'wb');
foreach ($stream as $record) {
    if ($record->tag() === 'INDI') {
        $record->node()->first('NAME')?->setValue('Анонім //');   // anonymise
    }
    fwrite($out, Gedcom::serialize($record->document()));         // unchanged records byte for byte
}
fclose($out);
$stream->diagnostics();

A streamed record's view knows only its own record: $record->view()?->familyXrefs() works, parents() throws RequiresDocumentException.

Indexed reading

Gedcom::open() reads the file once to index where each record is, then parses a record from disk when it is asked for; the last 1000 records are cached.

$doc = Gedcom::open('huge.ged');
echo count($doc), ' records', PHP_EOL;
echo $doc->find('@I1@/NAME')?->value(), PHP_EOL;
foreach ($doc->individual('@I3@')?->parents() ?? [] as $parent) {   // related records read from the file
    echo $parent->name()?->full(), PHP_EOL;
}

$report = Gedcom::validate($doc);           // record by record, the report of the whole document
$family = $doc->record('@F1@')?->document()->copy();   // an editable copy of one record
$whole = $doc->load();                      // or the whole document, as read() gives it

Records of an indexed document are read only: editing one throws ReadOnlyException.

Converting

use Korin\Gedcom\Version;

$losses = Gedcom::convertFile('huge.ged', 'huge-7.ged', Version::V7_0);
echo count($losses), ' losses', PHP_EOL;

The output and losses are those of convert(). Some rules need the whole file (pointers to records written later), so convertFile() reads it two to five times; memory stays at a few dozen MB for 50 MB.

Times and memory for 10 and 50 MB files are in the README.