# Overview

The [Data](https://docs.falkor.io/data/data-manager) feature allows **any data to be uploaded to the platform**. This data can then be linked to **integration points** - where applicable - on the platform. Typically, this data consists of **measurable data** to be used for **graphing, data tracking and activation**.

Data that is uploaded, should be **analytical** in nature and is not meant for mass data (see [Limits](https://developer.docs.falkor.io/data/overview#limits), below). When planning integrations, it is recommended to provide **summative data** instead of transactional. For example:

**Don't**\
Upload transactional sales data with rows detailing each purchase.

**Instead**\
Upload total sales for a user per row.

## Collections & Uploads

When data is uploaded it is assigned to a unique [data collection](https://docs.falkor.io/data/data-manager#data-collections). A collection could be considered a **container of multiple buckets** - and **each upload is 1 bucket in the collection**. A collection name is unique per account, and an upload is unique per collection.

The [Data](https://docs.falkor.io/data/data-manager#data-collections) feature has been designed this way in order to create agility when integrating multiple collections to other features and to structure uploads clearly.

When accessing the API endpoints you will always need to provide the `collection` and `upload` parameter to target the specific data that you are managing.

{% content-ref url="/pages/-MNPN0M6D1GNFDiZVohm" %}
[Upload](/falkor-dev-docs/data/upload.md)
{% endcontent-ref %}

## Data Processing

Data is processed as it is received. This allows several things to occur in near real-time:\
\
**Connects app users to data**\
Processing scans the uploaded data to identify users and to connect them to data. This allows for uploaded data to be integrated on a user level.\
\
**Connects data to locations**\
Processes locations to assign a location ID to a row. This allows time location filtering to occur on integrations.\
\
**Connects data to a timestamp**\
Processes dates to assign a timestamp to a row. This allows time-based filtering to occur on integrations.

**Optimizes Data**\
After data has changed in a collection, the platform optimizes & organizes data into a structure that allows the best performance in indexing and scanning.

**Notifies Integrations**\
As new data is received, integrations in platform features are notified - which invokes any built-in hooks to update data points on graphs or activities.

## Date Formats

| Format              | Example          | Parsed As        |
| ------------------- | ---------------- | ---------------- |
| DD/MM/YYYY          | 01/12/2020       | 01 December 2020 |
| D/MM/YYYY           | 1/12/2020        | 01 December 2020 |
| DD/M/YYYY           | 01/1/2020        | 01 January 2020  |
| D/M/YYYY            | 1/1/2020         | 01 January 2020  |
| DD MONTH YYYY       | 01 Feb 2020      | 01 February 2020 |
| DD LONG\_MONTH YYYY | 01 February 2020 | 01 February 2020 |
| YYYY-MM-DD          | 2020-03-01       | 01 March 2020    |

## Limits

| Type                       | Limit   |
| -------------------------- | ------- |
| **Cells Per Upload**       | 40,000  |
| **Uploads Per Collection** | 20      |
| **Cells Per Collection**   | 800,000 |
| **Collection Size**        | 34 MB   |

{% hint style="info" %}
**Cell Limits**\
Depending on the data, you can upload a maximum of 20 uploads per collection. Each upload can have up to 40,000 cells, which means a collection can have up to 800,000 cells. For example, if you have 10 columns each upload can have 4,000 rows. If you have 5 columns, the upload can have 8,000 rows.
{% endhint %}

## Reserved Types and Words

Specifically named headers allows data to be treated differently from other columns. This allows you to use "User Identifiers" when mapping collections to integration points on the platform.

| Header            | Description                                       |
| ----------------- | ------------------------------------------------- |
| **User Code**     | Processes the data in this column as a user code. |
| **User Mobile**   | Processes the data in this column as a mobile ID. |
| **User Email**    | Processes the data in this column as an email.    |
| **User Location** | Process the data in this column as a location.    |

## ISO Specifications

| Type              | Format                                                                                       |
| ----------------- | -------------------------------------------------------------------------------------------- |
| **Location**      | [ ISO 3166-1 alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) (e.g. Cape Town, ZA) |
| **Email**         | Valid email address                                                                          |
| **Mobile Number** | [ E.123 international notation](https://en.wikipedia.org/wiki/E.123) (No Spaces)             |

## Benchmarks

Data uploads can be intensive and have a **hard limit of 2,000 rows of data per upload**. When planning your integration it is important to be aware of upload time so you can **anticipate latency**. Data uploading happens **asynchronously** so you will only be able to call the endpoint once, per upload.

To give you an **idea of how long data takes to upload**, we have provided benchmarks based on estimated upload times when using the API.

| Data                  | Time (In Seconds) |
| --------------------- | ----------------- |
| 10 Columns, 1 Row     | 0.76              |
| 10 Columns, 10 Rows   | 1.06              |
| 10 Columns, 100 Rows  | 3.64              |
| 10 Columns, 1000 Rows | 31.06s            |
| 10 Columns, 2000 Rows | 60.95s            |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://everyday-digital.gitbook.io/falkor-dev-docs/data/overview.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.