Paclets and Paclet Development
Paclets and Paclet Development
This is a preliminary version of a tutorial on paclets and paclet development. In version 13 will be full Wolfram-style documentation in our products, but this document is intended to support developers until the final, complete documentation is available.
What are Paclets?
Paclets are units of Wolfram functionality, packaged up in a way that allows them to be discovered, installed, updated, and integrated seamlessly into the Wolfram environment. The essential element that makes a paclet is the PacletInfo.wl file, a small, simple file of metadata that describes the paclet, its requirements, and the ways in which it extends the Wolfram environment.
When in use, a paclet consists of a directory containing a PacletInfo.wl file along with one or more content files and subdirectories. A paclet can be compressed into a single .paclet file, which is a convenient form for distributing and installing, but it is not necessary for a paclet to be packed into a .paclet file.
The Paclet system was created years ago to provide a way for Wolfram to provide updates to data categories like CountryData, GraphData, CityData, etc. From the beginning, however, it was designed to handle arbitrary content, in particular Wolfram Language packages and all the supporting files that go along with them (documentation, libraries, palettes, and so on).
Paclets are used internally by the Wolfram system to break the product into components, and to provide updates and bug fixes outside of the normal product release cycle. The Mathematica 13 layout includes over 270 paclets.
We are now documenting and exposing the paclet system to users, so you can create and distribute your own products and packages. We are also creating a Paclet Repository that will accept user submissions, hoping to create a vibrant ecosystem of easily-discoverable and accessible paclets.
Changes beginning in version 12.1:
If you have done any programming with paclets in the past, pay attention to the “Changes beginning in version 12.1” boxes like this one, scattered throughout this document. They highlight things that have changed beginning in 12.1, which had a significant refactoring of the Paclet Manager, the component in the Wolfram Engine that handles paclets. The two biggest changes you should be aware of are that the Paclet Manager functions have moved from the PacletManager` context to the System` context, and that paclets are represented in the language as PacletObject[...] expressions now, not Paclet[...]. All the old PacletManager` context functions are still present, however, so any code that uses them will continue to work. You should explicitly use the PacletManager’ versions of the functions if you are writing code that must be compatible with 12.1 and also older versions.
If you have done any programming with paclets in the past, pay attention to the “Changes beginning in version 12.1” boxes like this one, scattered throughout this document. They highlight things that have changed beginning in 12.1, which had a significant refactoring of the Paclet Manager, the component in the Wolfram Engine that handles paclets. The two biggest changes you should be aware of are that the Paclet Manager functions have moved from the PacletManager` context to the System` context, and that paclets are represented in the language as PacletObject[...] expressions now, not Paclet[...]. All the old PacletManager` context functions are still present, however, so any code that uses them will continue to work. You should explicitly use the PacletManager’ versions of the functions if you are writing code that must be compatible with 12.1 and also older versions.
The PacletInfo.wl File
A PacletInfo.wl file is a simple file of metadata, in Wolfram Language format, that describes a paclet and the resources it provides. It is the presence of a PacletInfo.wl file that makes a directory or collection of files into a paclet. You can also name this file PacletInfo.m if you prefer.
Let’s examine a PacletInfo.wl file for a very simple paclet called, say, FileUtilities. This paclet is just a single implementation file, FileUtilities.wl, which defines the FileUtilities` context containing a few useful functions. Your users will load the package in their session in the usual way, using Needs[“FileUtilities`”].
PacletObject[<| "Name" "FileUtilities", "Version" "1.0", "Extensions" { {"Kernel", "Root" "Kernel", "Context" "FileUtilities`"} }|>]
The “Name” and “Version” fields are required in all paclets, and are the only required fields (although a paclet with only Name and Version would not be very useful). The “Extensions” field is where you describe what types of resources your paclet provides--the ways in which your paclet extends the system. A paclet that contains .wl or .m files that define packages to be loaded with Get or Needs should specify a “Kernel” extension and list the context(s) the paclet provides. In the above example, the Paclet Manager knows that when a user executes Needs[“FileUtilities`”] this paclet provides the .wl file that corresponds to that context.
Every paclet has a version number, and although the paclet system is designed to support side-by-side installations of multiple versions of a paclet, by default it uses the one with the highest version number. If you are releasing paclets to be used by others, make sure that you increment the version number with every release, otherwise your new version will not be recognized as being different from the old version.
You can put comments in Wolfram Language syntax into PacletInfo.wl files.
The essence of paclet functionality is that you declare the content your paclet provides via the PacletInfo.wl file, and when the paclet is installed those types of content will be found automatically by the system. Users generally don’t have to do anything beyond installing your paclet. Paclets “weave” themselves into the system so that their resources can be found. Palettes show up in the menu, documentation shows up in the Help Viewer, contexts can be loaded with Needs, etc.
You might be aware that the Wolfram Engine looks for many types of resources based on the value of $Path. The Paclet Manager has a much more powerful lookup mechanism that makes manual modifications to $Path a thing of the past. When the Wolfram Engine looks up a resource, $Path is still used, but the Paclet Manager is always consulted first, falling through to $Path only if no paclet provides the resource.
Changes beginning in version 12.1:
A significant change to the PacletInfo.wl file format was made in version 12.1. Formerly, the head of the expression was Paclet, not PacletObject, the contents were not wrapped in an Association, and the left-hand sides of rules were typically symbols (although they could always have been strings). Files in the old format are still supported, but files in the new format will not work in version of Wolfram products before 12.1. Here is what the previous PacletInfo.wl file looks like in the old format:
Paclet[
Name “FileUtilities”,
Version “1.0”,
Extensions {
{“Kernel”, Root “Kernel”, Context “FileUtilities`”}
}
]
A significant change to the PacletInfo.wl file format was made in version 12.1. Formerly, the head of the expression was Paclet, not PacletObject, the contents were not wrapped in an Association, and the left-hand sides of rules were typically symbols (although they could always have been strings). Files in the old format are still supported, but files in the new format will not work in version of Wolfram products before 12.1. Here is what the previous PacletInfo.wl file looks like in the old format:
Paclet[
Name “FileUtilities”,
Version “1.0”,
Extensions {
{“Kernel”, Root “Kernel”, Context “FileUtilities`”}
}
]
PacletInfo Fields
The PacletInfo.wl file supports a number of fields. Fields are always in the form “FieldName” -> value, where value is typically a string or list of simple elements like rules, lists, and strings. You cannot put arbitrary Wolfram Language expressions into field values; these files are not evaluated when read into the kernel, so you cannot put something in that evaluates to the value that you want--the file must contain the literal value.
◼
Name
The Name field is required. This is the name by which your paclet will be known. It is used in functions like PacletInstall and PacletFind. It is not legal to use a space or a hyphen (- character) in a paclet name, so developers typically use a single underscore (_) as a word separator if desired. The paclet name has no significance other than as a unique identifier. For example, it does not need to match the name of a context or file that your paclet provides. You can only include characters that are legal in a filename on every operating system, so avoid punctuation or special characters.
◼
Version
The Version field is required. The version is a string, not a number (i.e., “1.0”, not 1.0). The version can have up to five digit blocks (“1.0.0.0.1234”), and the version comparison is exactly as you would expect (“1.10” is greater than “1.9”). You cannot include non-numeric characters like “a”, although this feature will likely be provided in the future.
◼
WolframVersion
Use this field to specify what versions of Mathematica/Wolfram Engine/WolframOne your paclet is compatible with. If you leave this field out, it is assumed that you are compatible with all versions. If your paclet’s WolframVersion specification does not match the currently running instance, your paclet will be invisible in that session. You can use *, +, -, and comma characters in this specification, as follows:
"12+" | All versions from 12.0.0 onward | |
"12.1.0-" | All versions from 12.1.0 and earlier | |
"12.*" | Only version 12.xx | |
"12" | Only version 12.xx (same as 12.*) | |
"11.3,12.0" | Only versions 11.3 and 12.0 |
◼
SystemID
Use this field to specify what operating systems your paclet is compatible with. If you leave this field out, it is assumed that you are compatible with all systems. If your paclet’s SystemID specification does not match the currently running instance, your paclet will be invisible in that session. The value can be a single string like “MacOSX-x86-64”, or a list like {“Windows-x-86-64”, “Linux-x86-64”}.
◼
Loading
Paclets with a “Kernel” extension can specify how that their contexts should be loaded. The default is Manual, meaning that users need to call Get or Needs on your contexts to load them, but you can specify “Startup” to have your contexts loaded every time the kernel starts up, or Automatic to specify that your contexts should be autoloaded the first time one of your exported symbols is used. The details of this system are described later on. Most developers will leave this out and rely on the default Manual loading behavior.
◼
Updating
If you specify Updating -> Automatic, your paclet will auto-update itself the first time it is used in a session. If no update is available, this check is very fast and does not touch the network. Note that auto-updating only works for paclets with a Kernel extension, as the moment when updating is performed is when a context supplied by the paclet is first asked to be loaded. For paclets with other types of extensions, like FrontEnd or Documentation, update checking is not performed before resources from the paclet are used.
◼
Root
The Root field specifies where the paclet’s contents are located relative to the paclet’s location, which is defined to be the directory in which the PacletInfo.wl file resides. The default value for Root is “.”, meaning that the paclet’s contents should be looked for in the same directory that contains the PacletInfo.wl file, but if you prefer a different file organization, for example where the PacletInfo.wl file sits outside the main content directory, you can use the Root field to point to that content directory. Most developers will not use this field.
◼
Qualifier
The Qualifier field is used in complex situations where you have one paclet that should be broken into multiple paclets, typically for different operating systems. For example, you might have a paclet that includes a large executable or a large LibraryLink library, and there will be a different one for each SystemID. You could make a single paclet that contained subdirectories for each SystemID, but this might become very large, having copies of each binary for each system. You don’t want your users to have to download hundreds of megabytes of libraries for operating systems they are not using. Instead, you can split your paclet up into paclets “qualified” by the SystemID for which they are intended. Details of this scenario are described later on.
◼
Creator, Publisher, Description, Category, URL, Support
These fields have no specific meaning to the Paclet Manager, but they can be used to provide information to users about what the paclet is for, who created it, and where they can go for further information sand support.
◼
Custom fields
You can invent any field you like, as long as the name is a string and the value is a string or list of simple elements like strings, rules, and lists. Custom fields won’t be directly used by the Paclet Manager, but you can extract their values from Wolfram Language code.
Paclet Extensions
A paclet can extend the Wolfram environment in many different ways. Your paclet’s “Extensions” field tells the Paclet Manager the ways in which your paclet extends the Wolfram system.
The paclet system is fundamentally declarative--you declare, via the PacletInfo.wl file, what elements your paclet provides, and the Paclet Manager makes sure those elements can be found through normal system operation. In a sense, paclets are passive things; you don’t need to “load” a paclet into your session. You simply install it once, and from then on it weaves itself into the system so that the resources it provides can be found automatically.
When the Paclet Manager looks for a resource specified by an extension, it creates a full path based on the following components:
paclet location / paclet root / extension root / resource path
◼
The paclet location is the full path to the directory containing the PacletInfo.wl file. The Paclet Manager knows this path; it is not something you have to think about.
◼
The paclet root is the path, relative to the paclet location, that specifies where the paclet's main content resides. This is specified via the "Root" field at the top level of PacletInfo.wl. The default for this is ".", meaning that the contents are in the same directory, but if you want to create another level of directory hierarchy underneath the PacletInfo.wl file to hold the contents, you can specify that as the Root. Most developers will not use this.
◼
The extension root is the location within the paclet of the contents of that particular extension type. This is specified by a "Root" field within the extension. For most extensions there is a default root, so you do not have to specify a "Root" field explicitly as long as you use the usual conventions for paclet layout. For example, the default root for documentation is Documentation, and most paclet authors will use a Documentation subdirectory to hold their documentation notebooks. One exception is the "Kernel" extension. By convention, paclets should include their .wl and .m files in a Kernel subdirectory. However, "Kernel" is not the default root of a Kernel extension (for complex reasons of backward compatibility with older, non-paclet style layouts). This means that most developers will need to specifically include "Root" -> "Kernel" in their Kernel extensions.
◼
The resource path component is typically just the file name. For example, for a .wl file in a Kernel subdirectory, it is just the expected file name derived from the context (e.g., FileUtilities` --> FileUtilities.m). Some extensions let you specify an alternative relative path and/or filename for the actual file, if desired.
As discussed earlier, your paclet can have top-level “WolframVersion” and “SystemID” fields that specify its compatibility. Those two fields can also appear in extensions, to control which extensions are visible in particular systems. For example, if you have different versions of your .wl files depending on what SystemID is running, you can have multiple Kernel extensions that have different “SystemID” fields and different “Root” fields pointing at a different subdirectory for each system.
You can specify more than one type of each extension, with different properties like Root, SystemID, and WolframVersion.
Extension Types
Extension Types
Kernel
Kernel
Probably the most common resource a paclet can provide is a Wolfram Language package (a .wl or .m file or set of those files) that can be loaded with Needs or Get. This is specified via the “Kernel” extension.
PacletObject[<| "Name" "MyPaclet", "Version" "0.0.1", "Extensions" { {"Kernel", "Root" "Kernel", "Context" {"MyPaclet`"}}, ... other extensions }|>]
With a PacletInfo.wl file like the above, this is the layout that your paclet would be expected to have:
MyPaclet/
PacletInfo.wl
Kernel/
MyPaclet.m
PacletInfo.wl
Kernel/
MyPaclet.m
It is an intended, but not required, convention of paclets to have their .wl and .m files in a Kernel subdirectory. Unfortunately, “Kernel” is not the default Root of a Kernel extension, so if you use a Kernel subdirectory you will need to include “Root” -> “Kernel” in your extension specification, as shown above. If you want your .wl file(s) to sit next to the PacletInfo.wl file, not in a Kernel subdirectory, then you would not use a Root specification in your Kernel extension, relying instead on the default root of “.”.
It is not required that the context match the name of your paclet.
Subcontexts
Subcontexts
if your main .wl file defines MyContext`, and you have another .wl file\b, SomeSubcontext.wl, that defines the subcontext MyContext`SomeSubcontext`, then where do you place that SomeSubcontext.wl file in your layout so that it will automatically be found? In other words, where should the SomeSubcontext.wl file reside so that the lookup of MyContext`SomeSubcontext` will resolve to that file? The answer is that it can reside right next to the MyContext.wl file. The Paclet Manager supports resolving MyContext`SomeSubcontext` to a SomeSubcontext.wl file in your Kernel directory. You can think of MyContext` resolving to your Kernel directory by virtue of declaring MyContext` in your Kernel extension, and then the SomeSubcontext part resolves to the SomeSubcontext.wl file. If you have more deeply nested subcontexts like MyContext`SomeSubcontext`AnotherSubcontext`, then the AnotherSubcontext.m file needs to reside in a Kernel/AnotherSubcontext directory. The subcontext hierarchy needs to correspond to a directory hierarchy, except that the first part, MyContext` is “free”, mapping to the Kernel directory itself.
init.m files
init.m files
Many old-style applications use a special kernel feature for locating a file that corresponds to a context: If a context maps to a file location on $Path, but no appropriately-named .wl or .m file is found there, the kernel looks for a Kernel subdirectory with an init.m file in it, and reads that file. The init.m file typically just says something like Get[“MyContext`MyContext`”]. The double context will correctly map to a MyContext.m file in the MyContext parent directory. This is really just a hack to allow the context MyContext` to map to a MyContext.m file in a MyContext directory on $Path. This type of layout is supported in a paclet, but it is completely unnecessary and strongly discouraged. In the above example of a Kernel extension, there is no need to trick the system with a “double context”. The context MyContext` maps directly to the Kernel/MyContext.m file, no init.m file needed.
Documentation
Documentation
Paclets typically provide some documentation in Wolfram style, and if you have such documentation, then you declare a “Documentation” extension. The Documentation extension supports the standard “Root”, “SystemID”, and “WolframVersion” fields, but adds a “Language” fields. Most paclet authors will not need those fields, and instead rely on a conventional documentation layout:
PacletObject[<| "Name" "MyPaclet", "Version" "0.0.1", "Extensions" { {"Documentation"}, ... other extensions }|>]
With a PacletInfo.wl file like the above, this is the layout that your paclet would be expected to have:
MyPaclet/
PacletInfo.wl
Documentation/
English/
...
PacletInfo.wl
Documentation/
English/
...
If you have documentation for languages other than English you would have language-specific subdirectories alongside, or in place of, English.
The exact contents of the English directory are not specified above. The subject of authoring and building documentation is complex, and described more fully in a later section.
Fro\bntEnd
Fro\bntEnd
If your paclet has resources used by the notebook front end, like Stylesheets, Palettes, Bitmaps, TextResources, etc., it should declare a FrontEnd extension. Any stylesheets and palettes in your paclet will automatically show up in the appropriate places in the front end menus.
PacletObject[<| "Name" "MyPaclet", "Version" "0.0.1", "Extensions" { {"FrontEnd"}, ... other extensions }|>]
Although you can choose to specify a custom Root for this extension, most authors will use the default layout, which looks like:
MyPaclet/
PacletInfo.wl
FrontEnd/
Palettes/
StyleSheets/
SystemResources/
TextResources/
...
PacletInfo.wl
FrontEnd/
Palettes/
StyleSheets/
SystemResources/
TextResources/
...
This is the same layout as seen in the front end itself: $InstallationDirectory/SystemFiles/FrontEnd.
Path
Path
A paclet can declare that it has files that should be found by what look like path-based lookups, where the first part of the path is the paclet name.
Here is what your paclet layout might look like.
MyPaclet/
PacletInfo.wl
SomeFile.txt
...
PacletInfo.wl
SomeFile.txt
...
For any function that expects a file path (like Get, OpenRead, FindFile, etc.) you can now pass in “MyPaclet/SomeFile.txt”. The first part of the path must match the paclet name. If the SomeFile.txt file was in a subdirectory named Stuff, you would specify “MyPaclet/Stuff/SomeFile.txt”.
LibraryLink
LibraryLink
If your paclet has LibraryLink libraries, you specify a LibraryLink extension:
Most paclet authors will rely on the default layout expected by LibraryLink. Here is what your paclet layout would look like.
MyPaclet/
PacletInfo.wl
LibraryResources/
MacOSX-x86-64/
libMyLibrary.dylib
Windows-x86-64/
MyLibrary.dll
Linux-x86-64/
libMyLibrary.so
... for other SystemID values
...
PacletInfo.wl
LibraryResources/
MacOSX-x86-64/
libMyLibrary.dylib
Windows-x86-64/
MyLibrary.dll
Linux-x86-64/
libMyLibrary.so
... for other SystemID values
...
With this layout, LibraryLoad[“MyLibrary”] would automatically work. There is no need to modify $LibraryPath, the value that LibraryLink uses for locating libraries not located in paclets.
JLink
JLink
If your paclet has Java classes to be used with J/Link, you specify a JLink extension:
Most paclet authors will rely on the default root of the JLink extension, which is a “Java” directory. Here is what your paclet layout would look like.
MyPaclet/
PacletInfo.wl
Java/
myclasses.jar
moreclasses.jar
...
PacletInfo.wl
Java/
myclasses.jar
moreclasses.jar
...
With this layout, J/Link will automatically be able to find classes in your jar files. There is no need to call J/Link’s AddToClassPath function.
Asset
Asset
All the extension types that have been described so far are things that automatically “weave” themselves into the system. You don’t have to make a special paclet-aware call to, say, Needs in order to find a .wl file provided by a paclet. But there are many types of things that you might want to provide in a paclet for which there is no built-in handling in the Wolfram system. The Asset extension is a generic way to specify resources of arbitrary types that can be found by name using the “AssetLocation” selector.
Let’s say that you have a executable in your paclet that you will launch with StartProcess. The best way to do that is to use an Asset extension to assign a name to that executable, and then look up its path via the name. In this way you avoid any hard-coded paths in your package code. It is only in the PacletInfo.wl file that the resource name is mapped to an actual file location, and that is the only place that needs to be changed to modify the location or even the filename itself. It also makes it easy to provide multiple versions of the executable, each for a different SystemID or WolframVersion.
Here is an example of a paclet that contains executables for three different SystemIDs:
The Resources field within the Resource extension is a list of pairs that map string identifiers to string paths. The paths can have directory components in them, like “subdir/MyExe.app”.
Here is what your paclet layout might look like.
MyPaclet/
PacletInfo.wl
Mac/
MyExe.app
Windows/
MyExe.exe
Linux/
MyExe
...
PacletInfo.wl
Mac/
MyExe.app
Windows/
MyExe.exe
Linux/
MyExe
...
In your package code, when you want to obtain the full path to the executable so you can launch it, you use the “AssetLocation” selector on the PacletObject:
That’s all you need to do. The returned path will be different on different operating systems, and the code doesn’t need to change if you rename or move the executable within the paclet layout.
Changes beginning in version 12.1:
The “Asset” extension type was formerly called “Resource”, and the function that gave a resource path was PacletManager`PacletResource[pacletObj, “assetname”]. That function will continue to work in current and future versions of the PacletManager. Here is what the previous PacletInfo.wl file would look like using the older, but still compatible, “Resource” extension:
PacletObject[<|
“Name” “MyPaclet”,
“Version” “0.0.1”,
“Extensions” {
{“Resource”, “Root” “Mac”, “SystemID” “MacOSX-x86-64”, “Resources” {{“my_exe”, “MyExe.app”}}},
{“Resource”, “Root” “Windows”, “SystemID” “Windows-x86-64”, “Resources” {{“my_exe”, “MyExe.exe”}}},
{“Resource”, “Root” “Linux”, “SystemID” “Linux-x86-64”, “Resources” {{“my_exe”, “MyExe”}}},
... other extensions
}
|>]
The “Asset” extension type was formerly called “Resource”, and the function that gave a resource path was PacletManager`PacletResource[pacletObj, “assetname”]. That function will continue to work in current and future versions of the PacletManager. Here is what the previous PacletInfo.wl file would look like using the older, but still compatible, “Resource” extension:
PacletObject[<|
“Name” “MyPaclet”,
“Version” “0.0.1”,
“Extensions” {
{“Resource”, “Root” “Mac”, “SystemID” “MacOSX-x86-64”, “Resources” {{“my_exe”, “MyExe.app”}}},
{“Resource”, “Root” “Windows”, “SystemID” “Windows-x86-64”, “Resources” {{“my_exe”, “MyExe.exe”}}},
{“Resource”, “Root” “Linux”, “SystemID” “Linux-x86-64”, “Resources” {{“my_exe”, “MyExe”}}},
... other extensions
}
|>]
Custom Extensions
Custom Extensions
You can also write custom extensions of your own creation. One reason for doing this is to implement a “plug-in” type of system. For example, say you are writing a framework for digital filters written in the Wolfram Language. Your paclet supplies code for working with these filters, and perhaps it supplies some filters of its own, but you want other paclets to be able to provide filters that can be automatically found at run time. You can create a custom extension called, say, “DigitalFilters”, and paclets that supply such filters will use it:
This paclet would then have a Filters directory with one or more .wl files in it:
MyPaclet/
PacletInfo.wl
Filters/
SomeFilters.wl
...
PacletInfo.wl
Filters/
SomeFilters.wl
...
Your framework code can use the PacletManager`PacletResources function to locate all paclets that contain a DigitalFilters extension.
PacletManager`PacletResources returns a list of pairs: {{_PacletObject, {“/full/path/to/extension/root”}}, ...}. You can then extract the paths to all the extension roots, and enumerate and load all the .wl files (or any other files) in those directories.
If you are wondering why PacletResources is in the PacletManager` context and not the System` context, it is simply because we are not yet sure we have the correct design. Functionality for this purpose will eventually become part of the main interface of the Paclet Manager, but at this moment in time you can use PacletManager`PacletResources (and this function will be supported into the future).
Complex Example Paclet
Here is a hypothetical complex paclet that demonstrates many of the extension types and features discussed. If you can understand this paclet and how it matches up with the file layout, then you have a good understanding of paclet extensions.
Here is the matching file layout:
MyPaclet/
PacletInfo.wl
Kernel/
MyPackage.wl
MyOtherPackage.wl
Kernel120/
MyPackage.wl
MyOtherPackage.wl
LibraryResources/
Windows/
somelib.dll
Windwosx86-64/
somelib.dll
Documentation/
English/
...
FrontEnd/
Palettes/
MySuperPalette.nb
Resources/
images/
sun.jpg
moon.jpg
PacletInfo.wl
Kernel/
MyPackage.wl
MyOtherPackage.wl
Kernel120/
MyPackage.wl
MyOtherPackage.wl
LibraryResources/
Windows/
somelib.dll
Windwosx86-64/
somelib.dll
Documentation/
English/
...
FrontEnd/
Palettes/
MySuperPalette.nb
Resources/
images/
sun.jpg
moon.jpg
Installing and Updating
Paclets are typically obtained by downloading from a paclet server. Wolfram Research maintains a replicated set of paclet servers to supply paclets to users, and we are constructing a public Paclet Repository that will accept user submissions, making for a one-stop source for paclets, similar to package-management systems in other languages. Other means of distributing paclets, including setting up your own paclet server, are described later in this tutorial. Here, we discuss the handful of functions that deal with installing and updating paclets from servers.
PacletInstall
PacletInstall
PacletInstall is the function that installs and updates paclets. Here we ask to install a Wolfram paclet that is both in the Mathematica/WolframDesktop layout and also occasionally updated on the Wolfram paclet server:
Note that the result is a PacletObject expression, which is the Wolfram Language representation of a paclet. If there had been an update to the Interpreter paclet available on a server, that update would have been downloaded and installed by the above command. If a paclet is already present on your machine, and no update is known to be available, then PacletInstall returns without taking any significant time, so if you are writing code that uses another paclet, you can call PacletInstall on that paclet and not be concerned about doing something unnecessarily expensive on your system or those of your users.
If you want to ensure that a paclet is installed, but not actually trigger an update if one is available, you can use the AllowVersionUpdate -> False option.
Changes beginning in version 12.1:
In versions of the Wolfram Engine prior to the release version of 12.1, the functionality of PacletInstall was spread over two functions, PacletInstall and PacletUpdate. If you are writing code to run in 12.0 and earlier, you will need to use PacletUpdate to get updating behavior. PacletUpdate is still supported in 12.1 and later, although not documented. In effect, the behavior that was PacletUpdate in v12 and earlier became the behavior of PacletInstall in 12.1.
In versions of the Wolfram Engine prior to the release version of 12.1, the functionality of PacletInstall was spread over two functions, PacletInstall and PacletUpdate. If you are writing code to run in 12.0 and earlier, you will need to use PacletUpdate to get updating behavior. PacletUpdate is still supported in 12.1 and later, although not documented. In effect, the behavior that was PacletUpdate in v12 and earlier became the behavior of PacletInstall in 12.1.
Paclets are installed into a special are managed by the PacletManager, $UserBasePacletsDirectory:
The paclets themselves are installed into $UserBasePacletsDirectory/Repository. Although it is not the normal procedure, you can hand-copy paclet directories into the repository, and delete them as well. The changes will not be detected until the next kernel restart, but you can call RebuildPacletData[] to force the Paclet Manager to immediately detect any changes you have made manually.
Installing from a .paclet file or URL
Installing from a .paclet file or URL
PacletInstall[“PacletName”] is very convenient for installing and updating a paclet, but it only works for paclets that are deployed on a paclet server. Although it is very easy to establish your own paclet server (described later), you can also distribute paclets as .paclet files to be installed from the file system or from the web, including the Wolfram Cloud. A .paclet file is a packed version of a paclet directory, produced by the CreatePacletArchive function (described later). A .paclet file is what gets deployed to a paclet server, but you can also install them directly:
You can also manually copy a paclet directory into $UserBasePacletsDirectory/Repository. In this case, it will only be detected the next time the kernel is restarted, or you can call RebuildPacletData[] to force it to be detected immediately.
PacletSites
PacletSites
PacletSites is the function that returns the list of paclet sites known to your system. For the vast majority of users, this will be just one site, the Wolfram public paclet server:
You can use PacletSiteRegister and PacletSiteUnregister to manage this list of known paclet sites. If you try to remove the main Wolfram site, it will be restored at the start of your next session. The only reason to use PacletSiteRegister and PacletSiteUnregister is if you want your system to know about additional paclet servers, for example ones established by your organization. How to do that is discussed later in this document.
PacletSiteUpdate
PacletSiteUpdate
When you call PacletInstall to obtain or update a paclet, how does it know whether an update is available on a server? Users often assume that PacletInstall checks the server every time, and is therefore an expensive operation, but this is not the case. PacletInstall only checks a local cache of data about what paclets are on the servers. That local cache of data is updated by calling PacletSiteUpdate:
This operation can take a second or more, depending on your geographical location and download speed.
PacletSiteUpdate is called internally at various times, at least once every few days, and generally much more often than that. If you want to be sure that you have the latest data about what paclets are available, you can call PacletSiteUpdate yourself. If you know that a paclet has been deployed to a server only very recently, you can call PacletSiteUpdate to make sure your system knows about the new version.
As mentioned above, a call to PacletInstall generally does not trigger a call to PacletSiteUpdate, but one exception is if the paclet being requested is not installed locally, and not found in the cache of server data. In this case, PacletSiteUpdate is called internally to see if the paclet is available, rather than just failing based on potentially outdated cache information.
Finding Paclets
Paclets are designed to weave themselves into the system so that there is little need to operate on them directly. The Mathematica 13 layout, for example, has over 270 paclets in it, and they are silent components of the system; you don’t need to know any Paclet Manager functionality to use them. Sometimes, however, it is useful to inspect paclets on the system and perform operations with them, and this is often necessary for developers who are creating their own paclets.
PacletFind
PacletFind
PacletFind is the function that finds paclets that have been installed on the system. It returns a list of PacletObject expressions. PacletObject is the representation of a paclet in the Wolfram Language and can be used to inquire about various paclet properties. PacletFind takes a number of options that control what paclets it returns.
I have three different versions of the NeuralNetworks paclet on my machine.
The Paclet Manager only uses the one with the highest version number (and this list is in sorted order), but PacletFind returns all the paclets that are compatible with the current system. If you are wondering why there are three different versions, it is because one is in the Mathematica 13 layout, and the other two are in the local repository, having been downloaded as updates to paclets in older versions of Mathematica. These paclets show up in the output because they are still compatible with Mathematica 13, but only version 13.0.3 will be used because it has the highest version number.
Here we see the locations of the paclets, showing that the two older ones are in the local repository of installed paclets.
By default, PacletFind only returns paclets that are compatible with the current SystemID and version of the Wolfram Engine, but you can use the “SystemID” -> All and “WolframVersion” -> All options to show even ones that are not compatible.
You can use the * wildcard to see paclets with names that match a pattern:
Here we ask for all paclets whose names start with N and that have a Documentation extension:
PacletFindRemote
PacletFindRemote
PacletFindRemote is the function that finds paclets available on a paclet server.
This shows that there are several versions of NeuralNetworks available that are compatible, but none of these would be be downloaded if I did PacletInstall[“NeuralNetworks”] because they all have versions lower than the one I already have installed. PacletFindRemote is not a function that users have much reason to call, but paclet developers can use it to understand what is available on servers and determine if an update is available to a given paclet.
Getting Information About Paclets
There are several ways to get information about specific paclets. In most cases, the first step is to obtain the PacletObject expression that represents the paclet you want to know about. One way to get a PacletObject expression is to use the PacletFind function:
PacletFind returns a list in sorted order, so the paclet that is in use by the system, which is the one with the highest version number, is listed first. If you just want the active version of a paclet, you can use PacletObject[“pacletname”]:
PacletObject[“pacletname”] is essentially equivalent to First[PacletFind[“pacletname”]].
The Information function is an easy way to get a reasonably full set of data:
You can also call it with a selector:
Most extraction of information about a paclet is done with property extraction syntax applied to the PacletObject itself:
You can extract multiple properties at once:
The “Location” property can be useful, telling you where the paclet is found. Recall that the location of an installed paclet is defined to be the directory in which the PacletInfo.wl file resides.
Developing Paclets
Paclets are an ideal packaging format for almost any kind of Wolfram System functionality that is not simple enough to be a candidate for one of our specialized repositories (Function Repository, Data Repository, etc.) You can start developing paclets right away, and they will likely to be supported in versions of the Wolfram Engine going back to at least 9 (depending on precisely which features you use).
Start off by creating a directory for your paclet, say $UserDocumentsDirectory/MyPaclet. In this directory, create a PacletInfo.wl file with the following content. You can use the notebook front end or any text editor to create this.
Note that the actual name of the directory (MyPaclet) is not relevant. It could have any name, not necessarily matching the name in the PacletInfo.wl file. The paclet name comes from PacletInfo.wl, and it is only a common convention for the parent directory to have the same name.
Create a Kernel subdirectory of MyPaclet. This is the directory that will hold the MyContext.wl file that contains the function definitions that you want to provide. It is also not necessary for the name of the context (MyContext`) to match the paclet name, but it usually will.
Create a MyContext.wl file in the MyPaclet/Kernel directory and give it the following trivial package content:
BeginPackage["MyContext`"]
Squared
Begin["`Private`"]
Squared[x_] := x^2
End[]
EndPackage[]
Squared
Begin["`Private`"]
Squared[x_] := x^2
End[]
EndPackage[]
You have now created a paclet. To distribute this to others, you want to pack it up as a .paclet file (which is really just a zip file, but it has some special conventions, so always use CreatePacletArchive instead of another archiving utility). CreatePacletArchive produces a .paclet file next to the directory that it is given to pack:
Note that CreatePacletArchive creates a filename that mashes together the paclet name and version number. This precise filename is not required for the paclet, and it could be renamed to anything (still keeping the .paclet extension). It is what is in the PacletInfo.wl file that matters to the paclet.
You can now distribute the MyPaclet-1.0.paclet file to colleagues or deploy it to a paclet server.
To see it running in the current session, install it. This unpacks the .paclet file into $UserBasePacletsDirecory/Repository and makes it available to the system:
Load the paclet’s context and use it in the usual way:
For what follows, we want to uninstall the paclet. This deletes the paclet directory from the install location.
PacletDirectoryLoad and PacletDirectoryUnload
PacletDirectoryLoad and PacletDirectoryUnload
The section above was a walkthrough of the development and deployment process of a trivial paclet. If you are actively developing the code in a paclet, you probably don’t want to go through the process of packing it into a .paclet file and installing it just so that you can test your latest changes. This is where the PacletDirectoryLoad function comes in. PacletDirectoryLoad tells the Paclet Manager to look for paclets in a non-standard location. The directory you add will be searched for paclets, up to two levels deep, and these paclets will be made available to the system immediately. The added directory only applies for the current kernel session. You can think of PacletDirectoryLoad as the analog of PrependTo[$Path, “dir”] in the non-paclet world.
A typical workflow while developing a paclet is to create your files in the standard paclet layout and then edit them in place. You can call PacletDirectoryLoad[“/my/development/dir”] to let the Paclet Manager load your development version of the paclet. You only need to do this once in a session. Now you can edit your .wl files and call Get[“MyContext`”] repeatedly to reload them, picking up your newest changes.
If you modify your PacletInfo.wl file during development, say by adding a new extension type, you will need to call RebuildPacletData[] to let the Paclet Manager know it needs to rebuild its cache of paclet data.
A convenient feature of PacletDirectoryLoad is that if a paclet in the given directory has the exact same version number as one installed elsewhere, then the paclet from PacletDirectoryLoad will be used in preference to the other paclet. The rule can be stated as “PacletDirectoryLoad wins ties”. This means that if you are developing a paclet, and you have already installed version 1.0.0 of that paclet in your paclet repository, you can use PacletDirectoryLoad to point at your development version of the paclet without having to increment the version number in your development PacletInfo.wl file. It is absolutely critical that whenever you release a modification of any kind to a paclet, you must give it a higher version number than any previous version. But when you are doing development, it is convenient not to have to remember to increment the version number.
PacletDirectoryUnload removes a directory from the paclet search path, immediately detaching any paclets in those directories from the system. For example, Needs will no longer find contexts in those paclets, and any palettes or other front end resources will disappear from front end menus. Of course, if any code has already been loaded from a paclet, then PacletDirectoryUnload will not wipe that out of your session.
RebuildPacletData
RebuildPacletData
RebuildPacletData is the function you need to call whenever you alter paclets outside of normal Paclet Manager functions (like PacletInstall). If you are using Paclet Manager functions only, then the Paclet Manager keeps track of any changes, but if you make a change by hand, say by altering a PacletInfo.wl file, then the Paclet Manager has no way of knowing that something has changed and its cache might be out of date. Therefore, you need to call RebuildPacletData[], which causes the Paclet Manager to re-scan all the PacletInfo.wl files in the system. This only takes a second or two.
I often see people calling RebuildPacletData[] in cases where it is not necessary, and although it is safe to do this, you should understand what it does and that it is only necessary on occasions where you make manual changes to PacletInfo.wl files, or hand-copy paclet directories into or out of $UserBasePacletsDirectory/Repository.
Authoring and Building Documentation
Authoring and Building Documentation
Wolfram Workbench
Wolfram Workbench
Paclet Servers
It is easy to set up a paclet server on your own machine or for your company or organization. No logic is required on the server, and in fact a paclet “server” can be as simple as a directory visible on the network with a few special files in it. The Paclet Manager has tools to automatically create the required files.
Let's say that you have a set of .paclet files that you have built or been given by others, and you want to make them findable and installable by others in your organization. All you need is a directory that can be accessed by others on your network via a file: URL. Here I set up a paclet “server” in the PacletServer directory of my home directory. In this PacletSite directory I have created a Paclets subdirectory (it must have this name), and placed a handful of .paclet files in it. I call the PacletManager`BuildPacletSiteFiles function, pointing at the PacletServer directory:
This function examines all the .paclet files, extracts information from them, and creates two files in the PacletSite directory: PacletSite.m and PacletSite.mz. These are the index files that clients will read when you call PacletSiteUpdate (it’s actually the PacletSite.mz file, a compressed version, that is downloaded by clients). Once those two files have been created, you now have a functioning paclet site. You add it via a file: URL.
Commands like PacletInstall and PacletFindRemote will now include this paclet server when looking for the latest versions of paclets.
The procedure for setting up a webserver-based paclet site is very similar. You create the same directory structure, a parent directory with a Paclets subdirectory containing .paclet files, then call PacletManager`BuildPacletSiteFiles on that parent directory. You simply make this directory visible to your web server, and give people the URL to the parent directory, perhaps something like:
When new paclets are to be added to the server, just run BuildPacletSiteFiles again. If paclets are updates to older versions, delete the older versions from the Paclets directory before calling BuildPacletSiteFiles.