OPAM: A Package Management System for OCaml |
Opam is a source-based package manager for OCaml. It supports multiple
simultaneous compiler installations, flexible package constraints, and
a Git-friendly development work-flow.
A package management system has typically two kinds of users: end-users who install and use packages for their own projects; and
packagers, who create and upload packages. End-users want to
install on their machine a consistent collection of packages –
a package being a collection of OCaml libraries and/or programs.
Packagers want to take a collection of their own libraries and
programs and make them available to other developpers.
This document describes the design of Opam to answer both of these needs.
In this document, $home
, $opam
, and
$path
are assumed to be defined as follows:
User variables are written in capital letters, prefixed by $
. For
instance package names will be written $NAME
, package versions
$VERSION
, and the version of the ocaml compiler currently
installed $SWITCH
.
This document is organized as follows: Section ??
describes the core of Opam, e.g. the management of
packages. Section ?? describes how
repositories are handled, Section ?? focus on
compiler switches and finally Section ?? explain how
packages can define configuration variables (which can be later used
by the build system).
The client state is stored on the filesystem, under $opam.
All the configurations files, libraries and binaries related to a
specific instance of the OCaml compiler in $opam/$SWITCH
, where
$SWITCH
is the name of that specific compiler instance. See
Section ?? for more details about compiler
switches.
$opam/packages/$NAME/$NAME.$VERSION/opam
is the
specification for the package $NAME
with version
$VERSION
(which might not be installed). The format of opam
files is described in §??.$opam/packages/$NAME/$NAME.$VERSION/descr
contains the
description for the version $VERSION
of package $NAME
(which might not be installed). The first line of this file is the
package synopsis.$opam/packages/$NAME/$NAME.$VERSION/url
contains the
upstream location for version $VERSION
of package $NAME
(which might not be installed). The format of url files is
described in §??.$opam/packages/$NAME/$NAME.$VERSION/files/
contains the
eventual overlay files on top of the upstream sources, for version
$VERSION
of package $NAME
(which might not be
installed). This files are copied in the build directory before
building and installing a package.$opam/archives/$NAME.$VERSION
opam.tar.gz+ contains the source
archives for the version $VERSION
of package
$NAME
. This archive might be a bit different from the
upstream library as it might have been repackaged by Opam to include
the evenutal overlay files.$opam/packages.dev.
contains cached information for
development packages. Opam uses it on update to check which
package needs to be upgraded.Most of the files in the client and server states share the same syntax defined in this section.
(* ... *)
OCaml comment blocks and also #
which
discard everything until the end of the current line.(
)
[
]
{
}
:
”.
All structured files share the same syntax:
|
$opam/config
follows the syntax defined in §?? with
the following restrictions:
|
The field opam-version indicates the current Opam format.
The field repositories contains the list of Opam repositories.
The field switch corresponds to the current compiler instance.
The field cores is the number of parallel process that Opam will use when trying to build the packages.
$opam/packages/$NAME/$NAME.$VERSION/opam
follows the syntax defined in
§?? with the following restrictions:
|
$NAME
, the content of version is $VERSION
. Both fields are optional are they can
be inferred from the filename.$NAME.config
and $NAME.install
, see
§?? and §??).Each command and command argument is substituted (see §?? and §??, with the identifier X being equivalent to the string "%{X}%") and can be followed by an optional filter, whose evaluation will result in the command (or the command argument) being executed or not. Filter expressions are typed and must evaluate to a boolean and binary operations apply to version strings.
A typical example is OS-related filters, where we can choose to execute commands depending on the current OS:
build: [ ["mv" "Makefile.unix" "Makefile"] {os != "win32"} ["mv" "Makefile.win32" "Makefile"] {os = "win32"} [make] ]
"foo" {>= "1.2" & <= "3.4"}
depends is an AND formula, which means that top-level
&
are not mandatory. For instance,
"foo" {<= "1.2"} ("bar" | "gna" {= "3.14"})
has the following
semantic: “both any version of package
"foo" lesser or equal to 1.2 and either any version of package
"bar" or the version 3.14 of package "gna".”
The depopts field contains a OR formula over
package names, which means that top-level |
are not
mandatory. This field express optional dependencies that
Opam will not try to install. However, when installing a new
package it will check if it is an optional dependency of already
installed packages. If it is the case, it will re-install the
packages (and their transitive forward-dependency closure).
failure
is defined in the scope of the filter, and can
be used to print messages in case there was an error (typically, a hint on how
it can be resolved, or a link to an open issue). success
is also
defined as syntactic sugar for !failure
.os
and ocaml-version
variables. In case the filter is not valid,
the package is disabled. The os and ocaml-version fields are
deprecated, please use available instead in newly created packages.The syntax of url files follows the one described in §?? with the following restrictions:
|
src, archive, http, local, git, hg, darcs are the location where the package upstream sources can be downloaded. It can be:
#<SHA1>
or #<tag-name>
or
#<branch-name>
. Use git, hg or darcs.
When an end-user starts Opam for the first time, he needs to
initialize $opam/
in a consistent state. In order to do so, he
should run:
$ opam init [--kind $KIND] $REPO $ADDRESS [--comp $VERSION]
Where:
$KIND
is the kind of Opam repository (default is http);
$REPO
is the name of the repository (default is default); and
$ADDRESS
is the repository address (default is
http://opam.ocamlpro.com/pub
).
$COMP
is the compiler version to use (default is the
version of the compiler installed on the system).
This command will:
$opam/config
(as specified in
§??)$opam/$SWITCH/installed
file,
$SWITCH
is the version from the OCaml used to compile $opam
.
In particular, we will not fail now
if there is no ocamlc
in $path
.$opam/repo/$REPO
by running the
appropriate operations (depending on the repository kind).$opam/repo/$REPO/packages/
to
$opam/packages/
).$opam/repo/index
and for each version
$VERSION
of package $NAME
appearing in the repository,
append the line '$REPO $NAME $VERSION'
to the file.$opam/archives
,
$opam/$SWITCH/lib/
, $opam/$SWITCH/bin/
and $opam/$SWITCH/doc/
.When an end-user wants to have information on all available packages, he should run:
$ opam list
This command will parse $opam/$SWITCH/installed
to know the
installed packages, and
$opam/packages/$NAME/$NAME.$VERSION/opam
to get all the
available packages. It will then build a summary of each packages. The
description of each package will be read in
$opam/packages/$NAME/$NAME.$VERSION/descr
if it exists.
For instance, if batteries version 1.1.3 is installed, ounit version 2.3+dev is installed and camomille is not installed, then running the previous command should display:
batteries 1.1.3 Batteries is a standard library replacement ounit 2.3+dev Test framework camomille -- Unicode support
In case the end-user wants a more details view of a specific package, he should run:
$ opam info $NAME
This command will parse $opam/$SWITCH/installed
to get the
installed version of $NAME
, will process
$opam/repo/index
to get the repository where the package comes
from and will look for $opam/packages/$NAME/$NAME.$VERSION/opam
to get available versions of $NAME
. It can then display:
package: $NAME version: $VERSION versions: $VERSION1, $VERSION2, ... libraries: $LIB1, $LIB2, ... syntax: $SYNTAX1, $SYNTAX2, ... repository: $REPO description: $SYNOPSIS $LINE1 $LINE2 $LINE3 ...
When an end-user wants to install a new package, he should run:
$ opam install $NAME
This command will:
$NAME $VERSION
in $opam/$SWITCH/installed
.
If not, then:$opam/archives/$NAME.VERSION.tar.gz
$opam/repo/index/
(see
??) to get the repository
$REPO
where the archive is available and then ask the
repository to download the archive if necessary.Once this is done, the archive might become available in
$opam/repo/$REPO/archives/
. It it is, copy it in the global
state (in $opam/archives
). If it is not, download it upstream
by looking at $opam/packages/$NAME/$NAME.$VERSION/url
and add
the eventual overlay files located in
$opam/packages/$NAME/$NAME.$VERSION/files
. Note: this files
can be overwritten by anything present in
$opam/$SWITCH/overlay/$NAME.$VERSION/
.
$opam/$SWITCH/build/$NAME.$VERSION/
.$opam/$SWITCH/bin
in the path.$opam/$SWITCH/build/$NAME.$VERSION/$NAME.install
to install
the created files. The file format is described in §??.$opam/$SWITCH/build/$NAME.$VERSION/$NAME.install
in
$opam/$SWITCH/install/
and the configuration file
$opam/$SWITCH/build/$NAME.$VERSION/$NAME.config
in
$opam/$SWITCH/config/
.When an end-user wants to know what are the latest packages available, he will write:
$ opam update
This command will follow the following steps:
$opam/config
.$opam/config
, call the
corresponding update scripts. Then, look at the difference between
the digest of files in the global state and in each repositories,
and fix the discrepancies (while notifying the user) if necessary.
Here, the order in which the repositories are specified
is important: the first repository containing a given version for a
package will be the one providing it (this can be changed manually
by editing $opam/repo/index
later).$opam/$SWITCH/reinstall
accordingly (for each compiler
version $SWITCH
).$opam/$SWITCH/reinstall
will be reinstalled (or
upgraded if a new version is available) on the next opam
upgrade (see §??), with $SWITCH
being
the current compiler version when the upgrade command is run.When an end-user wants to upgrade the packages installed on his host, he will write:
$ opam upgrade
This command will:
$opam/$SWITCH/reinstall
will be reinstalled (or upgraded if a new
version is available). It will install each non-installed packages in
topological order, similar to what it is done during the install step,
See §??.$opam/$SWITCH/reinstall
.When the user wants to remove a package, he should write:
$ opam remove $NAME
This command will check whether the package $NAME
is installed,
and if yes, it will display to the user the list packages that will be
uninstalled (ie. the transitive closure of all forward-dependencies).
If the user accepts the list, all the packages should be uninstalled,
and the client state should be let in a consistent state.
Dependency solving is a hard problem and we do not plan to start from scratch implementing a new SAT solver. Thus our plan to integrate (as a library) the Debian depency solver for CUDF files, which is written in OCaml.
Opam is able to manage concurrent compiler installations.
Compiler descriptions are stored in two files:
$opam/compilers/$VERSION/$VERSION[+$TAG]/comp
contains the meta-data
for a given compiler.$VERSION
is the compiler version and
$TAG
an optional tag (such as 4.01+fp
).
The format of .comp file is described in ??.$opam/compilers/$VERSION/$VERSION[+$TAG]/descr
contains the description of that compiler. $VERSION
is the
compiler version and $TAG
an optional tag (such as
4.01+fp
). The first line of that line is used as synopsis and
is displayed with opam repository list.Switch-related meta-data are stored under $opam/$SWITCH/
:
$opam/$SWITCH/installed
is the list of installed
packages for the compiler instance $SWITCH
. The file format
is described in §??.$opam/$SWITCH/installed.root
is the list of installed
packages roots for the compiler instance $SWITCH
.
The file format is described in §??. A package
root has been explicitely installed by the user.$opam/$SWITCH/config/$NAME.config
is a
platform-specific configuration file of for the installed package
$NAME
with the compiler instance $SWITCH
. The file
format is described in §??.$opam/$SWITCH/install/$NAME.install
is a
platform-specific package installation file for the installed
package $NAME
with the compiler instance $SWITCH
. The
file format is described in §??.$opam/$SWITCH/lib/$NAME/
contains the libraries
associated to the installed package $NAME
with the compiler
instance $SWITCH
.$opam/$SWITCH/doc/$NAME/
contains the documentation
associated to the installed package NAME with the compiler
instance $SWITCH
.$opam/$SWITCH/bin/
contains the program files for all
installed packages with the compiler instance
$SWITCH
.$opam/$SWITCH/build/$NAME.$VERSION/
is a tempory folder
used to build package $NAME
with version $VERSION
,
with compiler instance $SWITCH
.$opam/$SWITCH/reinstall
contains the list of packages
which has been changed upstream since the last upgrade. This can
happen for instance when a packager uploads a new archive or fix the
opam file for a specific package version. Every package appearing in
this file will be reinstalled (or upgraded if a new version is
available) during the next upgrade when the current instance of the
compiler is $SWITCH
. The file format is similar to the one
described in §??.$opam/$SWITCH/pinned
contains the list of pinned
packages. The file format is described in §??.$opam/$SWITCH/overlay/$NAME.$VERSION/
contains overlay
files for the version $VERSION
of package $NAME
, for the
switch $SWITCH
. These possible overlay files are opam, url, descr or the directory files/. If one of this
file (or directory) is present, the file (or directory) will be used
instead of the global one.$opam/$SWITCH/packages.dev
contains cached information
for dev and pinned packages. Opam uses it on update to check which
package needs to be upgraded.The following configuration files: $opam/$SWITCH/installed
,
$opam/$SWITCH/reinstall
, and $opam/repo/$REPO/updated
follow a very simple syntax. The file is a list of lines which
contains a space-separated name and a version. Each line
$NAME $VERSION
means that the version $VERSION
of
package $NAME
has been compiled with the compiler instance
$SWITCH
and has been installed on the system in
$opam/$SWITCH/lib/$NAME
and $opam/$SWITCH/bin/
.
For instance, if batteries version 1.0+beta and ocamlfind version 1.2 are installed, then
$opam/$SWITCH/installed
will contain:
|
The syntax of comp files follows the one described in §?? with the following restrictions:
|
#<SHA1>
or #<tag-name>
or
#<branch-name>
. Use git, hg or darcs.
opam config
command (see §??).CAMLP4 [ "pp-trace" ]
: this will look for the
compilation flags for the syntax extension "pp-trace"
and expand
the camlp4 command-line accordingly. All the syntax extensions used
should be present in packages.For instance the file, 3.12.1+memprof.comp describes OCaml, version 3.12.1 with the memory profiling patch enabled:
opam-version: "1" name: "3.12.1" src: "http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz" make: [ "world" "world.opt" ] patches: [ "http://bozman.cagdas.free.fr/documents/ocamlmemprof-3.12.0.patch" ] env: [ CAML_LD_LIBRARY_PATH = "%{lib}%/stublibs" ]
And the file trunk-g-notk-byte.comp describes OCaml from SVN trunk, with no tk support and only in bytecode, and all the libraries built with -g:
opam-version: "1" name: "trunk-g-notk-byte" src: "http://caml.inria.fr/pub/distrib/ocaml-3.12/ocaml-3.12.1.tar.gz" configure: [ "-no-tk" ] make: [ "world" ] bytecomp: [ "-g" ] bytelink: [ "-g" ] env: [ CAML_LD_LIBRARY_PATH = "%{lib}%/stublibs" ]
$opam/$SWITCH/install/NAME.install
follows the syntax defined
in §?? with the following restrictions:
|
$opam/$SWITCH/lib/$NAME/
.
$opam/$SWITCH/bin/
.
$opam/$SWITCH/sbin/
.
$opam/$SWITCH/doc/$NAME/
.
$opam/$SWITCH/share/$NAME/
.
$opam/$SWITCH/etc/$NAME/
.
$opam/$SWITCH/toplevel
.
$opam/$SWITCH/lib/stublibs/
$opam/$SWITCH/man/man3
. You can change the sub-directory by setting
the right optional argument (for instance:
man: [ "foo.1" {"man1"} ]
.
$SRC { $DST }
, the tool asks the user if
he wants to install $SRC
to the absolute path $DST
.
General remarks:
doc: [ "_build/foo.html" {"foo/index.html"} ]
will copy the
given HTML page under $opam/$SWITCH/doc/$NAME/foo/index.html
.?
.$NAME.install
files, containing the existing files only.$opam/$SWITCH/pinned
contains a list of lines of the form:
<name> <kind> <path>
<name>
is the name of the pinned package<kind>
is the kind of pinning. This could be version
,
local
, git
or darcs
.<path>
is either the version number (if kind is
version
) or the path to synchronize with.
If the user wants to switch to another compiler version, he should run:
$ opam switch [-alias-of $COMPILER] $SWITCH
This command will:
$COMPILER
is not set, set it to $SWITCH
$opam/$COMPILER
directory.$COMPILER
in $opam/config
.$opam/compilers/$VERSION/$VERSON[+$TAG]/comp
, where
$COMPILER = $VERSION[+TAG]
If the file does not exists, the
command will fail with a well-defined error.--prefix $opam/$SWITCH
to
./configure
) and initialize everything in $opam/
in a consistent state as if “opam init
” has just been called.$opam/aliases
with the line
$SWITCH $COMPILER
Configuration files for the remote repository REPO
are stored in
$opam/repo/$REPO
. Repositories can be of different kinds
(stored on the local filesystem, available via HTTP, stored under git,
…); they all share the same filesystem hierachy, which is
updated by different operations, depending on the repository kind.
$opam/repo/$REPO/version
contains the minimum version of
Opam which can read that repository meta-data.$opam/repo/$REPO/config
contains the configuration
off the repository $REPO
. The format of repository config
files is described in §??.$opam/repo/$REPO/packages/$PREFIX/$NAME.$VERSION/opam
is
the opam file for the package $NAME
with version
$VERSION
(which might not be installed) with any possible
$PREFIX
. The format of opam files is described in
§??.$opam/repo/$REPO/packages/$PREFIX/$NAME.$VERSION/descr
contains the textual description for the version $VERSION
of
package $NAME
(which might not be installed) with any
possible $PREFIX
– it should be in the same loacation as the
corresponding opam file. The first line of this file is the
package synopsis.$opam/repo/$REPO/packages/$PREFIX/$NAME.$VERSION/url
contains the upstream location for the version $VERSION
of
package $NAME
(which might not be installed) with any
possible $PREFIX
– it should be in the same loacation as the
corresponding opam file. The format of url files is
described in §??.$opam/repo/$REPO/packages/$PREFIX/$NAME.$VERSION/files/
contains the overlay files for the version $VERSION
of
package $NAME
(which might not be installed) with any
possible $PREFIX
– it should be in the same loacation as the
corresponding opam file. The overlay files are added to the
build directory when a package is built and installed.$opam/repo/$REPO/archives/$NAME.$VERSION.tar.gz
contains the
source archives for the version $VERSION
of package
$NAME
. This folder is populated when a package needs to be
downloaded and that the given repository expose such a file. If the
file is not present, Opam will download the package from the
upstream sources.$opam/repo/index
follows a very simple syntax: each line of the
file contains a space separated list of words
$NAME.$VERSION $REPO+
specifying that all the version
$VERSION
of package $NAME
is available in the remote
repositories $REPO+
. The file contains information on all
available packages (e.g. not only on the installed one).
For instance, if batteries version 1.0+beta is available
in the testing repository and ocamlfind version 1.2
is available in the default and testing repositories (where default is one being used), then $opam/repo/index
will
contain:
|
An user can manage remote repositories, by writing:
$ opam repository list # 'opam repository' works as well $ opam repository add [--kind $KIND] $REPO $ADRESS $ opam repository remove $REPO
list
lists the current repositories by looking at
$opam/config
add [--kind $KIND] $REPO $ADDRESS
initializes
$REPO
as described in §??.url#hash
or url#name
where hash is a given
revision name and name it the name of a tag or a branch.remove $REPO
deletes $opam/repo/$REPO
and removes
$REPO
from the repositories list in $opam/config
.
Then, for each package in $opam/repo/index
it updates the link
between packages and repositories (ie. it either deletes packages or
symlink them to the new repository containing the package).Any file can be processed using generated using a special mode of opam which can perform tests and substitutes variables (see §?? for the exact command to run). Substitution files contains some templates which will be replaced with some contents. The syntax of templates is the following:
%{$NAME:$VAR}%
are replaced by the value
of the variable $VAR
defined at the root of the file
$opam/$SWITCH/config/NAME.config
.%{$NAME.$LIB:$VAR}%
are replaced by the
value of the variable $VAR
defined in the $LIB
section
in the file $opam/$SWITCH/config/$NAME.config
$opam/SWITCH/config/NAME.config
follows the syntax defined in
§??, with the following restrictions:
|
$NAME.config
contains platform-dependent information which can
be useful for other libraries or syntax extensions that want to use
libraries defined in the package $NAME
.
The definitions “IDENT: BOOL”, “IDENT: STRING” and “IDENT: [ STRING+ ]”, are used to defined variables associated to this package, and are used to substitute variables in template files (see §??):
%{$NAME:$VAR}%
will refer to the variable $VAR
defined at the root of the configuration file $opam/$SWITCH/config/NAME.config
.%{$NAME.$LIB:$VAR}%
will refer to the variable $VAR
defined in the library or syntax section named
$LIB
in the configuration file $opam/$SWITCH/config/$NAME.config
.Each library and syntax section defines an OCaml library and the specific compilation flags to enable when using and linking with this library.
The distinction between libraries and syntax extensions is only useful
at compile time to know whether the options should be used as
compilation or pre-processing arguments (ie. should they go on the
compiler command line or should they be passed to the -pp
option). This is the responsibility of the build tool to do the right
thing and the <kind> of sections is only used for documentation
purposes in Opam.
The available options are:
Opam contains the minimal information to be able to use installed libraries. In order to do so, the end-user (or the packager) should run:
$ opam config list $ opam config var $NAME:$VAR $ opam config var $NAME.$LIB:$VAR $ opam config subst $FILENAME+ $ opam config [-R] include $NAME+ $ opam config [-R] bytecomp $NAME.$LIB+ $ opam config [-R] asmcomp $NAME.$LIB+ $ opam config [-R] bytelink $NAME.$LIB+ $ opam config [-R] asmlink $NAME.$LIB+
list
will return the list of all variables defined
in installed packages (see §??)
var $var
will return the value associated to the
variable $var
subst $FILENAME
replace any occurrence of
%{$NAME:$VAR}%
and %{$NAME.$LIB:$VAR}%
as specified in
§?? in $FILENAME.in
to create $FILENAME
.
includes $NAME
will return the list of paths to include when
compiling a project using the package $NAME
(-R
gives
a result taking into account the transitive closure of
dependencies).
bytecomp
, asmcomp
, bytelink
and
asmlink
return the associated value for the section
$LIB
in the file $opam/$SWITCH/config/$NAME.config
(-R
gives
a result taking into account the transitive closure of all
dependencies).
This document was translated from LATEX by HEVEA.