tscs37's Blog

SECL Future Proposals

SECL has seen some nice development for the last months. Which is my way of saying “sometimes I commit a few patches”. But I nonetheless put out some nice features and in this post I wanted to express what’s going to happen in the future and what I’ve got waiting for the next release.

Dynamic Call Alias

There is a lot of typing in SECL. To remove the need to type a lot, the function regcalias will be included. It takes two parameters; a function and a string. After the function executes, the function will be available under the specified string.

!(regcalias merge "m")
(m (hello) (world))

// Result after Expanding

(hello world)

Macros

Macros will happen, most definitely.

There is a proposal in the git repository and it will rely on upmerge support(an internal proposal, so I won’t detail it), atleast partially.

The basic syntax looks as follows;

!(*defmac weather_report ?string:replace {name} {weather} .
    "Hello {name}" weather: {weather}
)

!(*mac weather_report "Max Mustermann" "sunny")
!(**mac weather_reprot {name}: "Alexa Abertwroth" {weather}: "rainy")

The result of the above SECL code will then be transformed into the following;

"Hello Max Mustermann"
weather: sunny

( "Hello Alexa Abertwroth" weather: rainy )

*mac and **mac behave basically the same but using two stars will put the expanded macro into an isolated maplist instead of merging with the parent maplist.

Variables

Variables are more complex but I think I’ve got the basis for a proposal once macros have been merged.

Variables will exist in two variants, global and scoped. A scoped variable will overwrite a global variable when present and is defined into a maplist. Global variables will be available in any context, even above the context they are declared in.

Variabes are expanded after all Macros and Functions have run. If the result of the expansion results in an executable SECL context, the process is repeated. The order in which variables are expanded is undefined and should not be relied upon.

!(define* VAR_DEF 8080) // VAR_DEF is a global variable
!(define VAR_VAL 9090) // VAR_VAL is a scoped variable
!(set VAR_DEF 8181) // Set value of VAR_DEF

Mapstructure-based Unmarshaller

The current SECL marshaller is… lacking. While it can deal with most Go structures fairly efficiently, it lacks the ability to unpack into slices and maps. Mostly because I’m lazy and this is complicated.

SECL maplists don’t have uniform types so the expression (5 "hellO") is entirely valid but not easily expressed as a go type. For that reason we have the query package which can address these. A compatible go structure would look like this in v0.9.6;

type Example struct {
    Number int `secl:"[0]"`
    String string `secl:"[1]"`
}

To upgrade the unmarshaller further I want to finally add a mode to unpack into slices and maps. For this I will utilize the github.com/mitchellh/mapstructure package, which allows unpacking of map[string]interface{} and []interface{} types into a struct. So basically exactly what I want.

My current proposal will add the tag secl_ms to a struct;

type Example struct {
    Number int `secl_ms:"_0"`
    String string `secl_ms:"_1"`
}

The list will be unpacked into a hidden map with the keys _<index> formatting. This would break maps that use this formatting but allow using the mapstructure package without much interference. Both notations can coexist;

type Example struct {
    Number int `secl_ms:"_0" secl:"[0]"`
    String string `secl_ms:"_1" secl:"[1]"`
}

Type Metadata

Types aren’t just a simple assignment of data into categories. Atleast it should not be. SECL types however are purposefully limited. To expand them in a backwards compatible way, metadata will be added to them.

This metadata can be added only via the type assertion =>type and =>T by prefixing a mapkey with @meta=.

!(=>T "hello" string @meta=thisismetadata:thisismetadatavalue)

Merge Extension

Merge will allow special behaviour using metadata and additional parameters.

The current proposal uses the two options allow-superset and overwrite-mode.

The first option will allow only true or false boolean values. True is default and allows new keys to be added to the resulting map. If set to false, no new keys can be added unless the metaoption @meta=allow-superset:true is set. This can make configuration files more robust and strict.

The second option configures how merge will handle overwriting and validating data. It can be set to true, false or meta. When set to true, values can be overwritten, if set to false, they cannot be overwritten. In all three modes the metatag @meta=overwrite-mode can override this setting. As a metatag it can also have the fourth value; required, which makes merge fail if the value was not overwritten.

Metatags will be honored in the order they appear, they cannot be overwritten in the current proposal.

Secret String

Sometimes you need to store a password or api key inside a configuration file.

To ensure the safety of these strings you can use the special type secret_string, normally not accessible without type assertion.

When not using any metatags, secret string has no advantages over string.

Proposed metatags are; @meta=min-size, @meta=complexity, @meta=hibp-enabled and @meta=hibp-source.

min-size will require the string to have a minimum number of utf8 characters.

complexity will use the zxcvbn package to check the strength and require a minium score value.

hibp-enabled and hibp-source will, when enabled, hash the string with sha1 and look them up in the password dump of HIBP. This will prevent weak or leaked passwords from being used. hibp-source can also be configured to a local file for a lookup that doesn’t touch the internet.