F# Literals on Steroids

F# Literals on Steroids

Mar 13, 2023

In this post, I will describe how to convert any value to a literal value (the main constraint of the JsonType provider) using .NET Interactive. I will begin by revisiting literals and existing literal type providers and end by introducing a playful literal function that adds literal attributes behind the scenes:

JsonProvider limitations

F# is an ideal language for data-rich programming, thanks partly to its powerful type providers. Type providers allow F# to seamlessly integrate with external data sources, providing type-safe access to data at compile-time.

That comes with certain limitations that exist for valid reasons but often prevent the exploration of structured yet extremely real-time (on-demand) data.

Entry parameters of type providers are pretty limited to only simple types and constant expressions. There has been a discussion to enable Type Providers to create types from types on fslang-design.

The motivation was to give a "functionality that currently only the compiler is capable of, and that is typically handled today by code generation, manual implementation or workarounds.". The drawback is "This would significantly increase the complexity of the implementation of the compiler. It's a very likely new source of bugs".

So, for now, we stay with workarounds, and this post is about a few of them.

Let's first check how to consume json and other Type Providers.

Using JsonProvider

For more clarity, let's put the entry JSON on a separate line

Now our driver is no longer a valid constant expression, but we can make it valid with [<Literal>] attribute.

but this will still only work for compile-constant values:

One way to omit such a constraint is to use special type providers that generate literal values for various common scenarios:

or

LiteralProviders package can generate literals from the compile-time environment.

In fact, generating such a literal inside Type Provider is not complicated:

That means we can provide literals and satisfy JsonProvider (and others) with yet another Type Provider.

This is exactly what I do in my GeojsonCloud provider

Literal function

But what if we don't want/need to write dedicated type providers or are just experimenting, exploring various data with a well-structured but highly dynamic nature?

We could just create or convert the value to the literal like that:

The implementation of a literal in F# involves taking the name and value of the literal, creating a raw code representation of it, and submitting it as a code for execution:

Since the code for creating a literal is considered a hack and could potentially distract from the main focus of our notebooks, it may be beneficial to hide it in a dedicated NuGet package, such as an F# kernel extension. With this extension, we can easily convert an existing value to a literal using a magic command syntax:

Consuming any object as a parameter in the Type Provider

Although we can convert any object to a literal this way, providers still only expect simple types as entry parameters. One workaround is to use serialization to create a literal on the fly and then deserialize it on the provider implementation side. Here I use JsonProvider and anonymous objects, but it can be any provider that works on real types behind the scene:

Is this technic only a toy? Maybe, but I have found it pretty useful for playing with OpenAI in my area of interest (geospatial, smart city).

It is good for explorations, domain discoverability, tests, and, I believe, much more.

Enjoy this post?

Buy Paweł Stadnicki a coffee

More from Paweł Stadnicki