LuaZDF - Lua Zero Dependency Functions

Problem

For the development of my little Lua library basexx[1] I needed a simple utility function that splits a string into an array table with substrings that have the same length. This function is not part of the Lua standard libraries[2], a quick search in the LuaRocks[3] repository was also unsuccessful. Other languages offer this kind of function in their standard library[4]. In my case, I ended up implementing the function by myself. This specific function is not rocket science by itself, but it is always easy to create bugs when implementing software, so I would prefer an implemented and tested function for this case.

Now it is easy to imagine that I need this function for other projects, so the question is how to share this and other similar functions. Therefore I introduce LuaZDF and will now explain the main design decision that I made in developing it.

Common Approaches

There are three common approaches I know to collect and distribute simple utility functions. I will describe them with a few words.

Big Module

One approach is to collect this kind of utility functions in a single module and to include this module in the project. The module would collect all kind of functions: io, string, color, etc - like Penlight[5] for example. For me, the size of this imaginary module is the biggest disadvantage of this approach, as I would need this module for just one function, but would demand that the rest does not bother me. Everyone who worked with big environments and libraries knows the hassle that follows by including big modules into a project. You will start to depend on the decision made by the library, like deep class structures and conventions.

Domain Module

You can reduce the size of one big module by creating single modules for each domain (io, string, color, etc), but they will still include a bunch of functions that are not required. My little Lua library basexx itself is this kind of module. A quick look at the projects[6] that are using basexx shows that just one or two functions from the module are actually used. The basexx module currently contains 14 functions, which means that around 90% of the functions are not required for those projects, thus adding unnecessary amounts of data to the project. Also I would have to add another module dependency for my imaginary string library that contains my util function. This leads to even more unused code and thus more unnecessary resources.

Single Function Module

Considering the downsides that large amounts of unnecessary code have on your project, the approach to create a single module for each function seems reasonable. The npm package manager[7] contains a lot modules that match this approach. The dependency tree that this approach generates can of course lead to new problems[8]. In generall, I am in favour of this approach, as you can specifically include those dependencies that you really need. The main downside of this is that you end up with a lot of dependencies that need to be managed, one for each function.

Zero Dependency Functions (ZDF)

For LuaZDF, the available functions will be implemented separately in single files for each function. LuaZDF is not library like the common approaches, it is more a function registry. This approach seems similar to the "Single Function Module" approach but has one major difference: you do not have to handle dependencies, as every function/file contains all necessary source code for it to function on its own. LuaZDF follows the principle that is best described with the following Go Proverb[9]:

A little copying is better than a little dependency.

This means that all functions work out of the box when you copy them into your project. I need to tell you one restriction in the second sentence, that is to say that not all dependencies are expendable. For example, it is hard in Lua to work on the file system without using LuaFileSystem[10]. But all functions inside the ZDF registry are independent and have zero dependencies among themselves.

Let me show you with an example what that means. The usefull function[11] dayofweek requires the function isleapyear to work. The DRY principle[12] would automatically lead to two seperate functions. That is not wrong, but it leads to abstraction that hides functionality and adds dependency.

With the ZDF approach I try to offer/collect all kind of useful functions that are easy to add into a project and have a small footprint regarding their dependencies and size.

Coding vs. Managing

You may still question yourself: "Why do you not just add single function modules to LuaRocks?" That was the first question from a friend, when I told him about the ZDF approach. For me the main problem with every package manager is the task of managing all the dependencies. I love to code, but everything that has to do with managing is a horror for me.

I talk about both sides here, the creation and use of packages. The convention over configuration[13] paradigm reduces a lot of work, but I have problems with creating the packages. LuaRocks for example already has a simple and easy way to create a package[14], but that does not help me much. I always struggle with things that feel like configuration. For me, creating and updating the rockspec file[15] is the part with the lowest level of joy.

The main problems on using packages often are the dependencies and their management. For example almost all package managers force you to define a required version for a package. My experience is that in most cases the packages are just defining the minimum version and ignore a maximum version where the code compiles or runs correctly. I unterstand the motivation but it breaks my code so often - it really hurts.

On the other side, I like approaches like stb[16] where you can just copy the code into your project, and have nothing to manage afterwards. Also, the code is viewable up front and is not hidden within a package structure. I don’t know how to discribe it better as that "you actually see the code". You also have the responsibility to include new versions of the used code to your project, as there is no automatic management of getting newer versions - I hope that the simple kind of the functions within LuaZDF do not require many updates.

Closing Words

I tried to give you a rough understanding about the ideas and decisions behind LuaZDF. If things are still unclear, check out the website[17] for more information or write a question on reddit[18].

Sources

PS: Lua Mailing List Thread