Sources

A source is the way that Flowerer understands where your data should be classified. A source specifies how you interface with out databases and define different rules that specify the quality of the storage and the life cycle of the data.

How do I get started?

In order to get started, just create a new source. By default, it will take some retention policies that can be changed at any moment, so do not worry about them initially. What matter more are the endpoints where you are going to send the data.

How does the data that you send look like:

The data that you are seding, needs to be somehow classified, that is why we want you to send your tags.

A good way of tagging your data is specifying the type of metric (CPU usage, Vehicle Position, Bidding price, Exchange Rate, Service Status), or the source of the data (Host Name, Sensor Location, Market), or any other value that may be important to group together all the data in the future while querying. Having many tags will not make the querying noticeably slower. The number of tags does not affect to the size of the final storage. No tags computes the same space than having 100 tags specified for the storage.

Apart of the tagging of the data, you need to give values to the sent data. As before, you can send as many values as you wish in JSON format. The backend will identify the type of each parameter and store them in the most efficient possible way so that querying or the space used will be always optimal. The keys of the values do not compute of the storage needs.

An example of a JSON document that we can injest is:

{
    "tags": {
        "plugin": "network",
        "unit": "bs",
        "host": "cl12a212"
    },
    "value": {
        "upload": 123121,
        "download": 7823434,
    }
}

Data counting

In the above document, the storage will count one data point of 8bytes (4 bytes per value item).

Endpoints

Flowerer allows you to send us the data in many different matters:

HTTP

You send JSON documents with the format previously indicated. The URL contains the identifier for the source. It allows to index the data as it arrives, or to make transformations so that you can use CollectD directly and Flowerer will automatically transform CollectD type into Flowerer schema.

MQTT

You send JSON documents with the format previously indicated. The topic where you publish your message indicates the source where the data will land.

TCP Socket

As a way of integrating with more lightweight protocols, we offer the possibility to send the data using TCP sockets, so that IoT devices can easily and quickly get integrated into the system. The payload should be serialized using Avro.

UDP Socket

If you are constrained in bandwidth but still need real time data, or your connection is not stable and you do not mind losing part of the traffic, you can also send the traffic over UDP. The data is handled in the same manner as TCP. Just the binary necessary data. Only 46 bytes plus payload!

Future Prometheus

We will integrate also with Prometheus, so you can simply expose your metrics to us, and we will retrieve and process them.