Specificatoins of Command Line Utilities of Kyoto Tycoon

This document describes how to use command line utilities. They are useful to manage database contents and to test the library and its applications.

1. ktserver : to run the database server.
2. ktutiltest : to test the utility functions.
3. ktutilmgr : miscellaneous utilities.
4. ktutilserv : to test the server tool kit.
5. kctimedtest : to test the timed database.
6. kctimedmgr : to manage the timed database.
7. ktremotetest : to test the remote database.
8. ktremotemgr : to manage the remote database.

ktserver

The command `ktserver' runs the server managing database instances. This command is used in the following format. `db' specifies a database name. If no database is specified, an unnamed on-memory database is opened.

ktserver [-host str] [-port num] [-tout num] [-th num] [-log file] [-li|-ls|-le|-lz] [-ulog dir] [-ulim num] [-uasi num] [-sid num] [-ord] [-oat|-oas|-onl|-otl|-onr] [-asi num] [-ash] [-bgs dir] [-bgsi num] [-bgc str] [-dmn] [-pid file] [-scr file] [-mhost str] [-mport num] [-rts file] [-riv num] [-plsv file] [-plex str] [-pldb file] [db...]

Options feature the following.

• -host str : specifies the host name of the server.
• -port num : specifies the port number of the server.
• -tout num : specifies the timeout in seconds.
• -th num : specifies the number of worker threads. By default, it is 16.
• -log file : specifies the path of the log file. By default, logs are written into the standard output.
• -li : sets the logging level "INFO".
• -ls : sets the logging level "SYSTEM".
• -le : sets the logging level "ERROR".
• -lz : sets the logging level "NONE".
• -ulog dir : specifies the path of the update log directory. By default, it is disabled.
• -ulim num : specifies the limit size of each update log file.
• -uasi num : specifies the interval of synchronization of update log files. By default, it is disabled.
• -sid num : specifies the server ID number.
• -ord : opens the database as a reader.
• -oat : opens the database with the auto transaction option.
• -oas : opens the database with the auto synchronization option.
• -onl : opens the database with the no locking option.
• -otl : opens the database with the try locking option.
• -onr : opens the database with the no auto repair option.
• -asi num : specifies the interval of auto synchronization. By default, it is disabled.
• -ash : does physical synchronization while auto synchronization.
• -bgs dir : specifies the path of the background snapshot directory. By default, it is disabled.
• -bgsi num : specifies the interval of background snapshotting. By default, it is 180.
• -bgsc str : specifies the compression algorithm of the snapshot. "zlib", "lzo", are "lzma" are supported.
• -dmn : switches to a daemon process.
• -pid file : specifies the file to contain the process ID to send signals by.
• -cmd dir : specifies the command search path for outer commands. By default, it is the current directroy.
• -scr file : specifies the script file for the scripting extention.
• -mhost str : specifies the host name of the master server of replication.
• -mport num : specifies the port number of the master server of replication.
• -rts file : specifies the file to contain the replication time stamp.
• -riv num : specifies the interval of each replication operation in milliseconds. By default, it is 0.04.
• -plsv file : specifies the shared library file of a pluggable server.
• -plex str : specifies the configuration expression of a pluggable server.
• -pldb file : specifies the shared library file of a pluggable database.

This command returns 0 on success, another on failure.

To finish the server process running on foreground, input `Ctrl-C' on the terminal. To finish the server process running as a daemon, send a termination signal such as SIGTERM by the `kill' command. If a daemon process catches SIGHUP, the server restarts and the log file is re-opened. Because thr current directory of a daemon process is changed to the root directory, paths of related files should be described as their absolute paths.

The naming convention of database name is the same as polymorphic database of Kyoto Cabinet. If it is "-", the database will be a prototype hash database. If it is "+", the database will be a prototype tree database. If it is ":", the database will be a stash database. If it is "*", the database will be a cache hash database. If it is "%", the database will be a cache tree database. If its suffix is ".kch", the database will be a file hash database. If its suffix is ".kct", the database will be a file tree database. If its suffix is ".kcd", the database will be a directory hash database. If its suffix is ".kcf", the database will be a directory tree database. Tuning parameters can trail the name, separated by "#". Each parameter is composed of the name and the value, separated by "=". If the "type" parameter is specified, the database type is determined by the value in "-", "+", ":", "*", "%", "kch", "kct", "kcd", and "kcf". All database types support the logging parameters of "log", "logkinds", and "logpx". The prototype hash database and the prototype tree database do not support any other tuning parameter. The stash database supports "bnum". The cache hash database supports "opts", "bnum", "zcomp", "capcnt", "capsiz", and "zkey". The cache tree database supports all parameters of the cache hash database except for capacity limitation, and supports "psiz", "rcomp", "pccap" in addition. The file hash database supports "apow", "fpow", "opts", "bnum", "msiz", "dfunit", "zcomp", and "zkey". The file tree database supports all parameters of the file hash database and "psiz", "rcomp", "pccap" in addition. The directory hash database supports "opts", "zcomp", and "zkey". The directory tree database supports all parameters of the directory hash database and "psiz", "rcomp", "pccap" in addition.

Furthermore, several parameters are added by Kyoto Tycoon. "ktopts" sets options and the value can contain "p" for the persistent option. "ktcapcnt" sets the capacity by record number. "ktcapsiz" sets the capacity by database size.

ktutiltest

The command `ktutiltest' is a utility for facility test and performance test of the utility functions. This command is used in the following format. `url' specifies the target URL. `rnum' specifies the number of iterations. `proc' specifies the name of the procedure to call. `name' and `value' specify a pair of the key and the value of an input parameter. `path' specifies the path of a repository.

ktutiltest http [-th num] [-get|-head|-post|-put|-delete] [-body file] [-ah name value] [-qs name value] [-tout num] [-ka] url rnum

Performs test of HTTP sessions.

ktutiltest rpc [-th num] [-host str] [-port num] [-tout num] proc rnum [name value ...]

Performs test of RPC sessions.

ktutiltest ulog [-th num] [-ulim num] path rnum [name value ...]

Performs test of update logging.

Options feature the following.

• -th num : specifies the number of worker threads.
• -get : uses the GET method.
• -head : uses the HEAD method.
• -post : uses the POST method.
• -put : uses the PUT method.
• -delete : uses the DELETE method.
• -body file : sends the entity body of the content the file.
• -ah name value : adds an HTTP header.
• -qs name value : adds a key/value pair to the query string.
• -tout num : specifies the timeout in seconds.
• -ka : use keep-alive connection.
• -host str : specifies the host name of the server.
• -port num : specifies the port number of the server.
• -ulim num : specifies the limit size of each update log file.

This command returns 0 on success, another on failure.

ktutilmgr

The command `ktutilmgr' is a tool of miscellaneous utilities, and to show the configuration. This command is used in the following format. `url' specifies the target URL. `proc' specifies the name of the procedure to call. `name' and `value' specify a pair of the key and the value of an input parameter. `path' specifies the path of a repository.

ktutilmgr date [-ds str] [-jl num] [-wf] [-rf]

Prints date information. By default, prints the current UNIX time.

ktutilmgr http [-get|-head|-post|-put|-delete] [-body file] [-ah name value] [-qs name value] [-tout num] [-ph] [-ec num] url

Performs an HTTP session.

ktutilmgr rpc [-host str] [-port num] [-tout num] [-ienc str] [-oenc str] proc [name value ...]

Performs an RPC session.

ktutilmgr ulog [-ts num] [-uw] [-uf] path

Prints update logs.

ktutilmgr conf [-v|-i|-l|-p]

Shows the configuration of Kyoto Tycoon.

ktutilmgr version

Shows the version information of Kyoto Tycoon.

Options feature the following.

• -ds str : specifies the datetime string.
• -jl num : specifies the jet lag.
• -wf : formats the output in W3CDTF.
• -rf : formats the output in RFC 1123 format.
• -get : uses the GET method.
• -head : uses the HEAD method.
• -post : uses the POST method.
• -put : uses the PUT method.
• -delete : uses the DELETE method.
• -body file : sends the entity body of the content the file.
• -ah name value : adds an HTTP header.
• -qs name value : adds a key/value pair to the query string.
• -tout num : specifies the timeout in seconds.
• -ph : prints response headers.
• -ec num : reports error if the response code is not the same as the expected.
• -host str : specifies the host name of the server.
• -port num : specifies the port number of the server.
• -ienc str : specifies the encoding of the input data.
• -oenc str : specifies the encoding of the output data.
• -ts num : specifies the maximum time stamp of already read logs. "now" means the current time stamp.
• -uw : waits for update forever.
• -uf : prints status of each update log file.
• -v : show the version number of Kyoto Tycoon.
• -i : show options to include the headers of Tokyo Tycoon.
• -l : show options to link the library of Tokyo Tycoon.
• -p : show the directory path of the commands.

This command returns 0 on success, another on failure.

ktutilserv

The command `ktutilserv' is a utility for facility test and performance test of the server tool kit. This command is used in the following format. `basedir' specifies the document root directory.

ktutilserv echo [-host str] [-port num] [-tout num]

Runs the single-threaded echo server.

ktutilserv mtecho [-host str] [-port num] [-tout num] [-th num] [-li|-ls|-le|-lz]

Runs the multi-threaded echo server.

ktutilserv http [-host str] [-port num] [-tout num] [-th num] [-li|-ls|-le|-lz] [basedir]

Runs the HTTP server.

ktutilserv rpc [-host str] [-port num] [-tout num] [-th num] [-li|-ls|-le|-lz]

Runs the RPC server.

Options feature the following.

• -host str : specifies the host name of the server.
• -port num : specifies the port number of the server.
• -tout num : specifies the timeout in seconds.
• -th num : specifies the number of worker threads. By default, it is 16.
• -li : sets the logging level "INFO".
• -ls : sets the logging level "SYSTEM".
• -le : sets the logging level "ERROR".
• -lz : sets the logging level "NONE".

This command returns 0 on success, another on failure.

kttimedtest

The command `kttimedtest' is a utility for facility test and performance test of the timed database. This command is used in the following format. `path' specifies the path of a database file. `rnum' specifies the number of iterations.

kttimedtest order [-th num] [-rnd] [-set|-get|-getw|-rem|-etc] [-tran] [-oat|-onl|-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-lv] path rnum

Performs in-order tests.

kttimedtest queue [-th num] [-it num] [-rnd] [-oat|-onl|-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-lv] path rnum

Performs queuing operations.

kttimedtest wicked [-th num] [-it num] [-oat|-onl|-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-lv] path rnum

Performs mixed operations selected at random.

kttimedtest tran [-th num] [-it num] [-hard] [-oat|-onl|-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-lv] path rnum

Performs test of transaction.

kctimedtest mapred [-rnd] [-ru] [-oat|-onl|-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-lv] [-tmp str] [-dbnum num] [-clim num] [-cbnum num] [-xnl] [-xpm] [-xpr] [-xpf] [-xnc] path rnum

Performs MapReduce operations.

kttimedtest misc path

Performs miscellaneous tests.

Options feature the following.

• -th num : specifies the number of worker threads.
• -rnd : performs random test.
• -set : performs setting operation only.
• -get : performs getting operation only.
• -getw : performs getting with a buffer operation only.
• -rem : performs removing operation only.
• -etc : performs miscellaneous operations.
• -tran : performs transaction.
• -oat : opens the database with the auto transaction option.
• -oas : opens the database with the auto synchronization option.
• -onl : opens the database with the no locking option.
• -otl : opens the database with the try locking option.
• -onr : opens the database with the no auto repair option.
• -ulog str : specifies the update log directory.
• -ulim num : specifies the limit size of each update log file.
• -sid num : specifies the server ID number.
• -dbid num : specifies the database ID number.
• -lv : reports all errors.
• -it num : specifies the number of repetition.
• -hard : performs physical synchronization.
• -ru : reuses the existing database.
• -tmp str : specifies the path of a directory for temporary storage.
• -dbnum num : specifies the number of temporary databases.
• -clim num : specifies the limit size of cache memory.
• -cbnum num : specifies the bucket number of cache memory.
• -xnl : executes with the no locking option.
• -xpm : executes with the parallel mapper option.
• -xpr : executes with the parallel reducer option.
• -xpf : executes with the parallel flusher option.
• -xnc : executes with the no compression option.

This command returns 0 on success, another on failure.

kttimedmgr

The command `kttimedmgr' is a utility for test and debugging of the timed database and its applications. `path' specifies the path of a database file. `key' specifies the key of a record. `value' specifies the value of a record. `file' specifies the input/output file. `dir' specifies the input/output directory. `src' specifies other database files.

kttimedmgr create [-otr] [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] path

Creates a database file.

kttimedmgr inform [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-st] path

Prints status information.

kttimedmgr set [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-add|-app|-rep|-inci|-incd] [-sx] [-xt num] path key value

Stores a record.

kttimedmgr remove [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-sx] path key

Removes a record.

kttimedmgr get [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-rm] [-sx] [-px] [-pt] [-pz] path key

Prints the value of a record.

kttimedmgr list [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-des] [-max num] [-rm] [-sx] [-pv] [-px] [-pt] path [key]

Prints keys of all records, separated by line feeds.

kttimedmgr clear [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] path

Removes all records of a database.

kttimedmgr import [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-sx] [-xt num] path [file]

Imports records from a TSV file.

kttimedmgr copy [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] path file

Copies the whole database.

kttimedmgr dump [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] path [file]

Dumps records into a snapshot file.

kttimedmgr load [-otr] [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] path [file]

Loads records from a snapshot file.

kttimedmgr vacuum [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] path

Eliminates regions of expired records.

kttimedmgr recover [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-ts num] path dir

Recover the database with update log data.

kttimedmgr merge [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] [-add|-app|-rep] path src...

Merge records from other databases.

kttimedmgr check [-onl|-otl|-onr] [-ulog str] [-ulim num] [-sid num] [-dbid num] path

Checks consistency.

kttimedmgr bgsinform file

Prints status information of background snapshot files.

Options feature the following.

• -otr : opens the database with the truncation option.
• -onl : opens the database with the no locking option.
• -otl : opens the database with the try locking option.
• -onr : opens the database with the no auto repair option.
• -ulog str : specifies the update log directory.
• -ulim num : specifies the limit size of each update log file.
• -sid num : specifies the server ID number.
• -dbid num : specifies the database ID number.
• -st : prints miscellaneous information.
• -add : performs adding operation.
• -app : performs appending operation.
• -rep : performs replacing operation.
• -inci : performs integer increment operation.
• -incd : performs real number increment operation.
• -sx : the input data is evaluated as a hexadecimal data string.
• -xt num : specifies the expiration time.
• -rm : removes the record.
• -px : the output data is converted into a hexadecimal data string.
• -pt : prints the expiration time also.
• -pz : does not append line feed at the end of the output.
• -des : visits records in descending order.
• -max num : specifies the maximum number of shown records.
• -pv : prints values of records also.
• -ts num : specifies the maximum time stamp of already read logs. "now" means the current time stamp.

This command returns 0 on success, another on failure.

ktremotetest

The command `ktremotetest' is a utility for facility test and performance test of the remote database. This command is used in the following format. `rnum' specifies the number of iterations.

ktremotetest order [-th num] [-rnd] [-set|-get|-rem|-etc] [-host str] [-port num] [-tout num] rnum

Performs in-order tests.

ktremotetest bulk [-th num] [-bin] [-rnd] [-set|-get|-rem|-etc] [-bulk num] [-host str] [-port num] [-tout num] [-bnr] rnum

Performs bulk operation tests.

ktremotetest wicked [-th num] [-it num] [-host str] [-port num] [-tout num] rnum

Performs mixed operations selected at random.

ktremotetest usual [-th num] [-it num] [-host str] [-port num] [-tout num] [-kp num] [-vs num] [-xt num] [-iv num] rnum

Performs mixed operations to simulate usual use cases.

Options feature the following.

• -th num : specifies the number of worker threads.
• -rnd : performs random test.
• -set : performs setting operation only.
• -get : performs getting operation only.
• -rem : performs removing operation only.
• -etc : performs miscellaneous operations.
• -host str : specifies the host name of the server.
• -port num : specifies the port number of the server.
• -tout num : specifies the timeout in seconds.
• -bin : uses the binary protocol.
• -bulk num : specifies the number of records in a bulk operation.
• -bnr : enables the no-reply option in the binary protocol.
• -it num : specifies the number of repetition.
• -kp num : specifies the number of patterns of keys.
• -vs num : specifies the size of each value.
• -xt num : specifies the expiration date.
• -iv num : specifies the interval of each operation in milliseconds.

This command returns 0 on success, another on failure.

ktremotemgr

The command `ktremotedmgr' is a utility for test and debugging of the remote database and its applications. `proc' specifies the name of a procedure. `params' specifies arbitrary parameters. `mhost' specifies the host name of the master server. `key' specifies the key of a record. `value' specifies the value of a record.

ktremotemgr report [-host str] [-port num] [-tout num]

Prints statistics of the server.

ktremotemgr script [-host str] [-port num] [-tout num] [-bin] [-swname str] [-swtime num] [-ssname str] [-ssbrd] proc [params...]

Calls a procedure of the scripting extension.

ktremotemgr tunerepl [-host str] [-port num] [-tout num] [-mport str] [-ts num] [-iv num] [mhost]

Sets the replication configuration.

ktremotemgr inform [-host str] [-port num] [-tout num] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-st]

Prints status information of a database.

ktremotemgr clear [-host str] [-port num] [-tout num] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str]

Removes all records of a database.

ktremotemgr sync [-host str] [-port num] [-tout num] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-hard] [-cmd str]

Synchronizes updated contents with the file and the device.

ktremotemgr set [-host str] [-port num] [-tout num] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-add|-rep|-app|-inci|-incd] [-sx] [-xt num] key value

Stores a record.

ktremotemgr remove [-host str] [-port num] [-tout num] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-sx] key

Removes a record.

ktremotemgr get [-host str] [-port num] [-tout num] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-rm] [-sx] [-px] [-pt] [-pz] key

Prints the value of a record.

ktremotemgr list [-host str] [-port num] [-tout num] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-des] [-max num] [-rm] [-sx] [-pv] [-px] [-pt] [key]

Prints keys of all records, separated by line feeds.

ktremotemgr import [-host str] [-port num] [-tout num] [-db str] [-sx] [-xt num] [file]

Imports records from a TSV file.

ktremotemgr vacuum [-host str] [-port num] [-tout num] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-step num] path

Eliminates regions of expired records.

ktremotemgr slave [-host str] [-port num] [-tout num] [-ts num] [-sid num] [-ux] [-uw] [-uf] [-ur]

Simulates a client of replication and prints update logs.

ktremotemgr setbulk [-host str] [-port num] [-tout num] [-bin] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-sx] [-xt num] key value ...

Store records at once.

ktremotemgr removebulk [-host str] [-port num] [-tout num] [-bin] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-sx] key ...

Remove records at once.

ktremotemgr getbulk [-host str] [-port num] [-tout num] [-bin] [-swname str] [-swtime num] [-ssname str] [-ssbrd] [-db str] [-sx] [-px] key ...

Retrieve records at once.

Options feature the following.

• -host str : specifies the host name of the server.
• -port num : specifies the port number of the server.
• -tout num : specifies the timeout in seconds.
• -swname str : waits for a signal by a named condition variable.
• -swtime num : specifies the timeout of signal waiting in seconds.
• -ssname str : sends a signal to a named condition variable.
• -ssbrd : switches signal sending to broadcasting.
• -bin : uses the binary protocol.
• -mport num : specifies the port number of the master server.
• -ts num : specifies the maximum time stamp of already read logs. "now" means the current time stamp.
• -iv num : specifies the interval of each replication operation in milliseconds.
• -db str : specifies the target database.
• -st : prints miscellaneous information.
• -hard : performs physical synchronization with the device.
• -cmd str : specifies an outer command for postprocessing.
• -add : performs adding operation.
• -app : performs appending operation.
• -rep : performs replacing operation.
• -inci : performs integer increment operation.
• -incd : performs real number increment operation.
• -sx : the input data is evaluated as a hexadecimal data string.
• -xt num : specifies the expiration time.
• -rm : removes the record.
• -px : the output data is converted into a hexadecimal data string.
• -pt : prints the expiration time also.
• -pz : does not append line feed at the end of the output.
• -des : visits records in descending order.
• -max num : specifies the maximum number of shown records.
• -pv : prints values of records also.
• -step num : specifies the number of steps.
• -sid num : specifies the server ID number.
• -ux : fetches update logs of the specified server ID number only.
• -uw : waits for update forever.
• -uf : prints status of each update log file.
• -ur : remove old update log files.

This command returns 0 on success, another on failure.