Function Resource

Function Repository Resource:

OllamaSynthesize

Source Notebook

Interact with local AI/LLM models via an Ollama server

Contributed by: Arnoud Buzing

ResourceFunction["OllamaSynthesize"][prompt]

generates a AI model response for the given prompt.

ResourceFunction["OllamaSynthesize"][prompt,images]

generates a AI model response for the given prompt and list of images.

ResourceFunction["OllamaSynthesize"][list]

generates a AI model response for the prompts and images in the list.

Details and Options

OllamaSynthesize requires the user to pre-install the Ollama software and the desired models. This function does not manage the installation of Ollama and its models and will not work without it.
The default model used by OllamaSynthesize is "Llava" and must be pre-installed by the user. The "Llava" model supports images.
Models that do support images, like "LLama3.2" will not work with images and a failure will be returned.
You can specify a desired model with the "OllamaModel" option. Examples of how to do this are under the "Options" example section below.
OllamaSynthesize launches a server process and uses API requests to get its responses. The server binary is chosen automatically for default installation locations, but for non-default installation locations you can give the "OllamaServerBinary" option to specify that non-default location on your machine.

Examples

Basic Examples (5) 

Try a basic question:

In[1]:=
ResourceFunction[
CloudObject[
  "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/OllamaSynthesize"]]["Hello, who was Albert Einstein?"]
Out[1]=

Ask a question about an image:

In[2]:=
Out[2]=

Ask another question about a different image:

In[3]:=
Out[3]=

A similar question with a different vision-enabled LLM model:

In[4]:=
Out[4]=

Mix the question and image(s) in a single list:

In[5]:=
ResourceFunction[
CloudObject[
  "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/OllamaSynthesize"]][{\!\(\*
GraphicsBox[
TagBox[RasterBox[CompressedData["
1:eJztXT2PFEcQRbYDR+gScizxAxCRQxyhyxDWxXwZEEjm+DjJOhIQhCBBQIog
IgRCMiCEBImUgIQUkZGu761VuHj7unu652OnZ/dJbe5u1rszXV1Vrz6697cz
2yfO/7Rv377rv+7958Tpf/64du307p8be79sXb5+8cLlc39tXt45d+Hctd/P
/Lz3x/N74++98cvemK0x+/Tp0+zWrVuzgwcPzjAlGxsbs1OnTs3/vsb4AFlB
RpAVD8hwLbfxALI4fPiwlJUfR48eXfatrjH7T15mB5uMr1+/LvuWVxoheeFv
r169mssHuuWv4e9rLAcheZ08efIHXQL/8NefPHmyxLtebSj/dfPmzYXXrfVs
HAA/bCIvgHnk+/fvh73ZNeZ6wvI6fvy4fC3sJ792jeHBPiwWdz1+/HjpXB9r
7MGDB3O/igEbsUpxorKJsednXwYZDgFwoFh8j4HrU4fiiSEfZq8PyRf/wq9B
BzDwc1dxG3QqJqtV4kNXrlxZsImxeWaOj3lM5UrwGvhGxAO59guvZ71e5bwM
ZMM6FrNzSsdKRtPcMvQllIvBOsB6w/3ev39/4dpUwVwC8xMD61ifsoMtDMkK
tpttAdvNqYJtWkzHFE9RNtJ4HAbsYcpuYp1wbBf6LNg8JWPIbxX0TNk5zBXW
N/61Yb/H7JPlIEPANayHmF/C5wChz4L9C+HFixcr4c/YLuaO7e3tIk6ItRKy
sUqmWBcpHsjvh3U0NcR8e2pgDruIx/AeKe6O66mcmLIXU8qjlfDmPucD9xOS
G/7ehFuyjk3JLqZiUuMP8BvQAePQ8C+YB/CJPmJVrIHStaF0bKicTN/guNmv
ZcWbhwbzDtxXk3tSOdIpIOTvSzlEH8B9sA1I6XRujrQWKP2y/oCxgWWA+4Q9
V3JQnBdrsHaoPAJ81ljXYiwv5vMlirekcqQ1QD0/5DX250pxWsiOfVhTfjl2
sA+rpW8U9wh+mhN7xPIjtUD1CNQgLw/Yg+fPnydz0rE6X03g58TvNSOU65qK
TYzVkGsHdEr56NrBHLiJjllvQQ0+T8mty/6PL1++zN68eTN7+vTp/OchwP4b
PiEFHxPUIDeOOdvayG/fvs3l9PDhw9nVq1e/j5cvX3Z30xEwD27C7TneqSEm
4Ocsrb1ALjdu3Pgup93d3bmOvX37di7LvsG12pzcG3PNUC/qWMD32zQ/afj4
8ePszp0732UFHfvw4cMgcvJg/pFbk+A+mLHXDTn+tjp3CrCDJqvbt2/P5bcs
tJEZ6h5Ncq7WI4DX+r4P/A7fOWStkXNzTWwD7tHkhZ+H1isGy6wJD8Z9h/JF
PocHW9S0Vor/b4h91rl9O/BT5rPevXu3cB3ygx+Dj4txRswFbBDWCAbWa+la
zfVnTWpqGLm5JI41+pQd93SFPsv0C/L6/PnzwnXYS89F4NvUe8T6MTBPJc/K
7xl7D1WT9zWa0N5BvM56QW1greGeVR0c79Ek5igBP4Na79CbkLygT57j4+fX
r18vvEeqT9Cv99xn5WeI+WXIBHMP7qE4F8srVCPleQrZ0T72PrCt4Log+EVI
dyAv444xLlLSn5az5xUybsMdDbyuQj0WxjXNd5o9tWdluTfhdsZz8NoUf+f7
5LkymXB8DN9l1+7evRvkIsrWmO+wvSShXGhTH6dq9E05sIe/z1itw8+Zfa7n
LuqsClUjx+tNP/39p+KNmJ6BZ5gOMYyP4FqMO3L8E8oTQVfa1GFVP0yOb/Tx
quIxPjep1gj7UbzGy43fM7afLJUvZX7kZWZ+ClzQAzbR7GUqp5iz/4RtXI6d
U71nOb7Exz28zr2M8DysQ5hjpUecHzMbFvIVeNYm/WAx3ggeqPTIOCR0LQaO
nZrklVgvm9g5tTcJI4fLeD1VslbrycvY9g2ybvv3tfXHcTFz19Szsj3xAEdU
vOLRo0dzmaXyH6X5PBVDxZ5J+cPcWq6fR2Wb/LMY51frhOfQz7FdY7uZ47tL
91XAHqoYza5BlrjONqCpreJnivnBnHMiYvAyUbkFmyt/TfErNYf+Nb6Pyv+9
aT0oJ65JAXwF/NHXYo4dO1bsX1RvGNdLVH26TU+B/zzWa7++1bk8sTlXMlPP
mMq7dVmPt5ibB8ssN2eufLVfx6rPqk0NzOssy8DbOPNf3obHuJW/R74/9h8x
G9nVvoqQvDC2traK/JmHqqvjfVTvadscH3N47mUN5SpxJlYIft2F9Mh8Y0wG
SsdK+qI95/e1GXAs/Hv27NkFPSgB5kTNVVc20YPXvdc3yNSvdcg3Fnuzzyrd
u6J4cWl91tdlLNfo4wJwEfZJJbZLcZKudcwj1TOD+7HYOgTen9hm74p69tLn
vXfv3g8yU/E1x+ylZ+fF6iN99DKqONFqZCGfY7kodZZISa1JnReJ0aYn1csL
nFGB/U7b+VU60FfNI1SXMTlg/WNOMWKvK60NqjXadh8M20UF9p25fScKXdjb
HITi59QInRHRFCr/2ha+xokcfwis3230gnM3Q+4jNrsX862YVzuHoi3U+U5t
wf4slN/nXEWbeR5Tf5s/lyzFSUpQsg80BeaNqkek689mmdW+lyIFXu9tn9fX
sGM+DWA9L9W1VZNZH+cjeZ8Wy/V3FceX9MbVDuYDbWNRzl2FOD+Qe2ZiCEPz
xmWD563t+fDgHaxrIV6o/FpJrzWvu6mf68q58jb1FwN64zjvGLKRKr+QayPb
7jWoDVzn7GpPAe9jgu6FekVYT0psJMe4Yzz7oyv0xbtgI5HL93JDnK3kpuqZ
uWsn1Sc1JfTJldFPwL4tJLcubCRkD1s/ZbsI9B3fcMwWk5uykWPfI7sMlPbV
5AD9kCw36B/3jisbOaVzCLsC50K64I0KSm6qh1z1fEydu+diyO+Cgl6xf7Mc
l7eVqs9i/b1i/2PoHAJkw3zSbKXpnOodmMp5M23B/GMo3wG52Z4MxU/AW5Rv
W3OS5X2vkAH5EmUrrff/2bNnCzYydG4H/LDVF6cK5euXsYahc9bvr8bm5ubC
fXI8ws/SVz/IMqF6UZZddwKvVH4O48iRIwtyM33K/X6eWqHONhzD2oTOhfqQ
Dx06JGMAdX7vFPMgqgYyplqh4iiXLl2aHThwYEFuPKaoYwala33F06Vg2UFu
+/fvD8pr6rlhQO05HiO87NDzH5Lb2NZcH+ij96pPQHbwuzs7O1JuqxK7sa7V
Et/wHvdVsIsGfvYxcZEYxtSLugzU+B2Q3BMx5dw/fBh0C37BztZqei7ZmMD7
a6bEPyAjPE/ovDM1apBZ6oynGgFdKv0uvRqef2oya3rOXs1ym4rMQns3mRPb
9x+C12OE9k+NOVabisxC+4shR3COWO5UfX/rmGOeIXqP+oaqh0F/cnQFPpDl
Ntb1y1x/2bWkEqjzIkpqEl3uwewTJWfUjQm85trUkFQOcqz1KL7PGmIUA/eW
ta0hsV8c6/eJd3mm2tBgrth2jnkuxsof2YeP1Y4rxM6jHcP79YXaakkeXfIG
1d8zVn8GdLWXfmio85JLz3ypbQ64JlNLLQkIfde5nQfZBCrnVUNPjD/zqqbv
kE2djwd9Aa/CumRbh9+VzMeuYx7qjO0awGdApoad6x+6VuMc1IrSc8/W8lo+
7NyzHPm1Pa9uje4A+WGfOWIv+D0f1+DnJt+d8C83sSxf
"], {{0, 102.}, {109., 0}}, {0, 255},
ColorFunction->RGBColor,
ImageResolution->{72, 72}],
BoxForm`ImageTag["Byte", ColorSpace -> "RGB", Interleaving -> True],
Selectable->False],
DefaultBaseStyle->"ImageGraphics",
ImageSizeRaw->{109., 102.},
PlotRange->{{0, 109.}, {0, 102.}}]\), "What is in this image?"}]
Out[5]=

Scope (1) 

Solve basic math problems with step by step instructions:

In[6]:=
ResourceFunction[
CloudObject[
   "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/OllamaSynthesize"]][
  "I have three boxes of six pencils each. How many pencils do I have?"] // Style[#, "Text"] &
Out[6]=

Options (3) 

The default model is "Llava". Use the "OllamaModel" option to specify another one:

In[7]:=
ResourceFunction[
CloudObject[
  "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/OllamaSynthesize"]]["Hello, who are what are you?", "OllamaModel" -> "llama3.2"]
Out[7]=

Larger models work too, but are slower. Also your machine will need to have sufficient GPU memory to run the model:

In[8]:=
ResourceFunction[
CloudObject[
  "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/OllamaSynthesize"]]["Hello, who are what are you?", "OllamaModel" -> "gemma2:27b"]
Out[8]=

If you specify a model that does not exist or a model that you did not download locally, then an error is raised:

In[9]:=
ResourceFunction[
CloudObject[
  "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/OllamaSynthesize"]]["Hello, who are what are you?", "OllamaModel" -> "xxx"]
Out[9]=

Possible Issues (3) 

Repeated calls to OllamaSynthesize will give slightly randomized results and sometimes these results can be wildly incorrect:

In[10]:=
ResourceFunction[
CloudObject[
   "https://www.wolframcloud.com/obj/arnoudbuzing/DeployedResources/Function/OllamaSynthesize"]][
  "Two train stations called A and B are 50 miles apart. A train leaves from A towoards B at 20mph. Another train leaves from B to A at 30 mph. How long after they leave do they pass each other?"] // Style[#, "Text"] &
Out[10]=

You can use Solve to do this correctly. The result is given in hours:

In[11]:=
Solve[Quantity[20, "Miles"/"Hours"] Quantity[x, "Hours"] == Quantity[50, "Miles"] - Quantity[30, "Miles"/"Hours"] Quantity[x, "Hours"], x]
Out[11]=

You are responsible for installing Ollama on your machine and for the downloading of the models you wish to use. To list the available models, you can run the following (undocumented) command:

In[12]:=
ollamaModels[]
Out[12]=