plugindatabase.json / LoxBerry::System::PluginDB

LoxBerry uses a local plugin database of your installed plugins, using a json file. As plugin developer, it is allowed to read the plugin database. But refrain from changing the plugin database from your plugin.

This article explains the data model of the plugin database.

For Perl and PHP, dedicated functions exist to read data from the plugin database. See Perl-Modul LoxBerry::System (Perl) and PHP Module loxberry_system.php (PHP).

Additionally the Perl library LoxBerry::System::PluginDB is explained, that is used as internal library for LoxBerry-Core for plugin management.

LoxBerry 2.x

This information is valid for LoxBerry V2.0 and above. LoxBerry below 2.0 used a plugindatabase.dat that was a line-based, pipe-seperated text file.

Location of the plugin database

Default installation /opt/loxberry/data/system/plugindatabase.json
Using LoxBerry::System $LoxBerry::System::PLUGINDATABASE (this is a global variable)
Using loxberry_system.phpPLUGINDATABASE (this is a global constant)
Using other languages Read the OS environment variable $LBSDATA
$LBSDATA/plugindatabase.json

Data model

Level 1Level 2 Level 3 Level 4 Description Example
plugins Contains an object list of all installed plugins
<md5hash> This is the unique key of the plugin. It is the md5-Hash of "<author_name><author_email><name><folder>" from your plugin.cfg  "f21873f370693…."
author_email The authors email address "git@loxberry.de"
author_name The Plugin author name "C.Woerstenfeld"
autoupdate 0 … Plugin does not provide automatic updates

1 … Updates disabled

2 … Notify about Updates

3 … Automatic Updates to release versions

4 … Automatic Updates to prerelease versions
"4"
epoch_lastupdated Unix epoch timestamp when the user or AutoUpdate last updated the plugin

undefined if the plugin was never updated, or last update was in a LB version below LB2.0
1568375268
epoch_firstinstalled Unix epoch timestamp when the user did the initial install of the plugin

undefined if the plugin was first installed in a LB version below LB2.0
1568375102
folder The installation folder of your plugin. Usually it is equal to NAME, but on uniqueness problems, it will be different "miniserverbackup"
interface This is the interface version of the plugin interface (currently 1.0 or 2.0) "2.0"
loglevel Log Level 0-7 (Tabelle siehe unten)

If set to -1, this indicates PLUGINDB_LOGLEVELS_ENABLED = 0
"5"
loglevels_enabled 0 … Plugin does not use the loglevel setting from the user in plugin management

1 … Plugin uses the loglevel setting from plugin management
1
md5 Same as Level 2 md5 "f21873f370693…."
name The short name of your plugin "miniserverbackup"
orig_folder Normally this field is undefined. 

If 'name' or 'folder' is used by another plugin, 'folder' and 'name' are uniquified by LoxBerry, therefore storing the plugins original folder/name in these attributes.
orig_name Normally this field is undefined.

If 'name' or 'folder' is used by another plugin, 'folder' and 'name' are uniquified by LoxBerry, therefore storing the plugins original folder/name in these attributes.
prereleasecfg URL to the plugin's Pre-Release Config file. This file contains the new version and the download url for the ZIP archive for new prereleases. Used in conjunction with AutoUpdate (see above) "https://raw.githubusercontent.com/…."
releasecfg URL to the plugin's Release Config file. This file contains the new version and the download url for the ZIP archive for new releases. Used in conjunction with AutoUpdate (see above) "https://raw.githubusercontent.com/…."
title The common title of your plugin "Miniserver Backup"
version The version of the plugin "2019.6.5.1"
directories This is an object list of the plugin's directories
lbpbindir "/opt/loxberry/bin/plugins/miniserverbackup"
lbpconfigdir "/opt/loxberry/config/plugins/miniserverbackup"
lbpdatadir "/opt/loxberry/data/plugins/miniserverbackup"
lbphtmlauthdir "/opt/loxberry/webfrontend/htmlauth/plugins/miniserverbackup"
lbphtmldir "/opt/loxberry/webfrontend/html/plugins/miniserverbackup"
lbplogdir "/opt/loxberry/log/plugins/miniserverbackup"
lbptemplatedir "/opt/loxberry/templates/plugins/miniserverbackup"
installfiles This folder contains plugin's install *.sh files, and apt and dpkg directories  "/opt/loxberry/data/system/install/miniserverbackup"
files This is an object list of files used by the plugin.
daemon "/opt/loxberry/system/daemons/plugins/miniserverbackup"
sudoers "/opt/loxberry/system/sudoers/miniserverbackup"
uninstall "/opt/loxberry/data/system/uninstall/miniserverbackup"

Example with one plugin:

Example json file Quelle erweitern

{
    "plugins": {
        "f21873f370693661863909d413dbdc50": {
            "author_email": "git@loxberry.woerstenfeld.de",
            "author_name": "M.Schlenstedt und C.Woerstenfeld",
            "autoupdate": "4",
            "directories": {
                "lbpbindir": "/opt/loxberry/bin/plugins/miniserverbackup",
                "lbpconfigdir": "/opt/loxberry/config/plugins/miniserverbackup",
                "lbpdatadir": "/opt/loxberry/data/plugins/miniserverbackup",
                "lbphtmlauthdir": "/opt/loxberry/webfrontend/htmlauth/plugins/miniserverbackup",
                "lbphtmldir": "/opt/loxberry/webfrontend/html/plugins/miniserverbackup",
                "lbplogdir": "/opt/loxberry/log/plugins/miniserverbackup",
                "lbptemplatedir": "/opt/loxberry/templates/plugins/miniserverbackup",
                "installfiles": "/opt/loxberry/data/system/install/miniserverbackup"
            },
            "files": {
                "daemon": "/opt/loxberry/system/daemons/plugins/miniserverbackup",
                "sudoers": "/opt/loxberry/system/sudoers/miniserverbackup",
                "uninstall": "/opt/loxberry/data/system/uninstall/miniserverbackup"
            },
            "folder": "miniserverbackup",
            "interface": "2.0",
            "loglevel": "5",
            "loglevels_enabled": 1,
            "md5": "f21873f370693661863909d413dbdc50",
            "name": "miniserverbackup",
            "prereleasecfg": "https://raw.githubusercontent.com/Woersty/LoxBerry-Plugin-miniserverbackup/master/prerelease.cfg",
            "releasecfg": "https://raw.githubusercontent.com/Woersty/LoxBerry-Plugin-miniserverbackup/master/release.cfg",
            "title": "Miniserver Backup",
            "version": "2019.6.5.1"
        }
    }
}

Monitor changes of plugins

If your plugin (e.g. a running daemon) requires to monitor, if a plugin was installed, updated or removed, instead of continuously reading the plugindatabase.json, you should prefer to monitor our change tracking file: plugins_state.json - Monitor plugin installations, updates, uninstalls

Performance considerations

Take a note on the use-case you are willing to do. This is a performance ranking depending:

1. JSON pure

For read-only access, that are 99% of all use-cases, directly read the json file:

JSON

use LoxBerry::System;
use JSON;
my data;
eval {
    $data = JSON::from_json( LoxBerry::System::read_file( $LoxBerry::System::PLUGINDATABASE ) );
};
if ($@) {
    print "JSON invalid or empty";
}
 
foreach my $plugin ( keys %{$data->{plugins}} ) {
    print "Title: " . $plugins->{$plugin}->{title};
}

2. LoxBerry::JSON

(uses JSON)

For reading and changing settings of already installed plugins. It's killer features compared to pure json are failure handling, easy writing and change detection to avoid writing to the SD if nothing has changed.

JSON

use LoxBerry::JSON;
$dbobj = LoxBerry::JSON->new();
$data = $jsonobj->open(filename => $LoxBerry::System::PLUGINDATABASE );
 
foreach my $plugin ( keys %{$data->{plugins}} ) {
    print "Title: " . $plugins->{$plugin}->{title};
    $plugins->{$plugin}->{title} = "New title";
}
$dbobj->write();

3. LoxBerry::System::PluginDB

(uses LoxBerry::JSON)

Especially for creating new plugins, and it is thought to work with one plugin. It has no capability to handle the full database, but gives you the data of one plugin to read and modify. 

It cuts off some tasks as searching for plugin attributes, it automatically handles adding/updating/removing a plugin, and it automatically fills some json attributes like the directories and files section.

See the examples in the LoxBerry::System::PluginDB section.

LoxBerry::System::PluginDB

This is an object class, accessing the plugindatabase.json, to add, modify and delete entries.

Include the library with 

use LoxBerry::System::PluginDB;

Add or update a plugin

Create a plugin object, no matter if you add or update an entry. The returning object tells you, if the object is new or not.

# Create the object. You can use atrributes you like, there is no check against attribute names.
# To create a new plugin, mandatory fields are author_name, author_email, name, folder.
# Do NOT send the md5 property. This is calculated automatically.
 
my $plugin = LoxBerry::System::PluginDB->plugin(
        author_name => $oldplugin->{PLUGINDB_AUTHOR_NAME},
        author_email => $oldplugin->{PLUGINDB_AUTHOR_EMAIL},
        name => $oldplugin->{PLUGINDB_NAME},
        folder => $oldplugin->{PLUGINDB_FOLDER},
        version => $oldplugin->{PLUGINDB_VERSION},
        title => $oldplugin->{PLUGINDB_TITLE},
        interface => $oldplugin->{PLUGINDB_INTERFACE},
        autoupdate => $oldplugin->{PLUGINDB_AUTOUPDATE},
        releasecfg => $oldplugin->{PLUGINDB_RELEASECFG},
        prereleasecfg => $oldplugin->{PLUGINDB_PRERELEASECFG},
        loglevel => $oldplugin->{PLUGINDB_LOGLEVEL},
        loglevels_enabled => $oldplugin->{PLUGINDB_LOGLEVELS_ENABLED},
);
 
# Check, if the command was successful
if (!$plugin) { 
    print "ERROR! Could not create plugin!"; 
    exit;
}
 
# Check if this was an update or a new plugin
if ($plugin->_isnew) {
    print "This is a new plugin";
} else {
    print "This is an updated plugin";
}
 
# You now can access the md5 hash, that is already set in a new plugin:
print "The plugin's unique id is " . $plugin->md5;

You now have a plugin object for further modification:

if($plugin->_isnew) {
    $plugin->installed_at(time());
}
$plugin->updated_at(time());

The directories and files list in a plugin is calculated automatically on save. You don't need to fill these properties.

On the end, you can save the plugin:

$plugin->save;

This will calculate files and folders lists and adds/updates the plugin in the plugindatabase.json.

Change attributes of a plugin

This is quite the same as adding/updating a plugin, but it is easified by using the md5 as key:

# Get the plugin with it's md5 hash
my $plugin = LoxBerry::System::PluginDB->plugin( md5 => 'abcdefg' );
if(!$plugin) {
    print "ERROR! Plugin not found!"; 
    exit;
}
 
# You may change attributes in two different ways, as you prefer:
$plugin->loglevel(5);
$plugin->{loglevel} = 5;
 
# For save, you can use brackets or not, makes no difference.
$plugin->save; # OR
$plugin->save();

Remove a plugin

This also works like a charme:

# Get the plugin with it's md5 hash
my $plugin = LoxBerry::System::PluginDB->plugin( md5 => 'abcdefg' );
if(!$plugin) {
    print "ERROR! Plugin not found!"; 
    exit;
}
 
# And now we remove it
$plugin->remove();

Remove automatically saves, no additional →save required.

Search a plugin by name / folder

In some situations, you may not have the md5 hash in your hands to get the plugin. The lib has a search function included, returing the results as array of md5 hashes.

The search is non-case-sensitive! 

You can query every property of a plugin, therefore use the property and needle as parameters (e.g. name ⇒ 'wiringpi' searches for the name property).

# Search for the folder name
my @result = LoxBerry::System::PluginDB->search( folder => 'wiringpi' );
# The resultset is an array of strings with md5 hashes.
 
# If you only expect one element you may assign the result to a scalar with brackets:
my ( $result ) = LoxBerry::System::PluginDB->search( folder => 'wiringpi' );
 
# With the md5 hash, you now can fetch the plugin
my $plugin = LoxBerry::System::PluginDB->plugin( md5 => $result );
if(!$plugin) {
    print "ERROR! Plugin not found!"; 
    exit;
}

Search with "or" and "and"

# Search for a plugin, that has interface = 2.0 AND has autoupdate enabled. The result is an array.
my @result = LoxBerry::System::PluginDB->search( autoupdate => '1', interface => '2.0' );
# The default operator is AND
 
# Search for a plugin, that has name OR folder = wiringpi
my ( $result ) = LoxBerry::System::PluginDB->search( folder => 'wiringpi', name => 'wiringpi', _condition => 'or' );
This can be used as a more fuzzy variant to search for a plugin.

Using a custom plugindatabase.json

This feature is for reading the "secured" plugindatabase.json. Use the parameter _dbfile => "<full path to file>" (take note of the underscore)

Side note: LoxBerry creates a copy of the original plugindatabase.json as plugindatabase.json.secured that is read-only for user loxberry. This copy is used to read the Plugin AutoUpdate url's, and secured that a non-priviledged user cannot modify the update url's.

# Get the plugin using a different dbfile
my $plugin = LoxBerry::System::PluginDB->plugin( 
    md5 => 'abcdefg',
    _dbfile => "$lbsdatadir/plugindatabase.json.secured"
 );
if(!$plugin) {
    print "ERROR! Plugin not found!"; 
    exit;
}
 
my $releasecfg = $plugin->releasecfg;
 
undef $plugin1;

The ->search function also needs the _dbfile parameter to search in the custom file. 

The other methods (save, remove) don't need the _dbfile parameter, as the plugin was already initialized from that file.

Additional things to know

  • As you request the plugin by the →plugin method, the plugin database is in RAM. If you wait a long time until →save, other processes may have changes the plugin database on disk, overwriting their changes.  
  • This does not happen, if you edit multiple plugins in one process (script), because a process shares the database connection to it's db file. Examples:
    • You have queried two plugins in your script and changed attributes in both plugins. You call →save on both plugins → All changes are written.
    • You have queried two plugins in your script and changes attributes in both plugins. You call →save on one plugin → Only the attributes of the →save plugin are saved.
    • You have two scripts, changing different plugin attributes in parallel → Last writer wins and may overwrite settings from the other.
  • You can use every attribute name to store it in the database. Only the method names are protected (so you cannot create an attribute called 'save' or 'plugin',…).
  • Every attribute beginning with an underscore (e.g. _isnew) is NOT saved to the database. Thereby you can use the plugin object as temporary variable store, e.g.
    • Set a variable: $plugin->_tempdir("/tmp/plugininstall"); or $plugin->{_tempdir} = "/tmp/plugininstall";
    • Get the variable: $plugin->_tempdir or $plugin→{_tempdir};
  • When you are finished working with a plugin, you should undef it (undef $plugin) explicitely, as all handles stay in memory, also after save.