Table sets in the console¶
The concept behind knowledge is explained in Knowledge.
A table set holds the tables in your documents -- worksheets, CSV and JSON files, tables inside a Word file and, when your deployment parses PDFs, the tables of a PDF -- each with its columns and rows, so you can ask which rows meet a condition and what they add up to. The Table sets segment of the Knowledge section is where you create a table set, have it filled from your collections, look through its tables and ask a table a question.
The list of table sets sits on the left and the one you picked on the right. Search above the list to narrow it, and use the plus to create one.
You need an administrator account of the organisation, and the knowledge capability has to be on
in the organisation's ceiling. You do the same with a key in
Table sets through the API.
Add a table set¶
Select the plus above the list. The Add table set drawer asks for:
| Field | What you enter |
|---|---|
| Name | unique within the organisation. Policy names the table set by this name, so it cannot change later |
| Description | what is in the table set |
| Labels | your organisation's own classification (below) |
| Metadata | optional. Your own key and value pairs, which no rule reads |
| Collections | the collections the table set reads its sources from, each with an optional filter |
| Ingest pipeline | how a source is read into tables. Leave it empty to use Tables: documents |
Select Add and the table set opens. When its collections already held sources, the screen asks you to read them in (Sources read).
A table set's screen¶
At the top are the name, the description and Read again. You edit the description where it stands. Below that, from top to bottom:
| Part | What you do there |
|---|---|
| Tables | find a table and open it |
| Labels | classify the table set for policy |
| Metadata | your own key and value pairs, which no rule reads |
| Collections | choose which collections the table set reads from |
| Ingest pipeline | the steps that read a source into tables |
| Sources read | the status per source |
| Run history | the runs in which sources were read again |
| Delete table set | delete the table set with all its tables and rows |
The description, the labels, the metadata, the collections and the pipeline are saved together: as soon as something has changed, a bar with Save appears.
Fill the table set from collections¶
Collections¶
Under Collections, tick the collections the table set reads from. Under a ticked collection is Filter: keys and values a source's metadata must carry. Leave the filter empty and the whole collection is read.
Ingest pipeline¶
Choose a built-in tables template, or an ingest pipeline of your own that ends in the Table Store step. None of the templates calls a model:
| Template | What it reads |
|---|---|
| Tables: structured | worksheets, CSV and JSON files |
| Tables: documents (the default) | the same, plus the tables inside a Word file, a slide deck, a web page or Markdown |
| Tables: advanced parse | the same, plus the tables of a PDF or a scan, through the deployment's parsing service |
Below it is the drawing of the steps. When the pipeline does not end in Table Store, the warning No Table Store is shown: nothing then reaches the table set, and Core refuses such a pipeline. How Core reads a worksheet into columns and types is explained in How a table is read.
Sources read, and reading again¶
Sources read shows per source the collection, the status (Queued, Working, Ready or Failed), the number of Tables and Rows, an error code and the moment of the last change. Search by title or filter by status. The title is a link to the source in its collection. A source that is Ready with zero tables holds no table, such as a PDF of prose. The error codes are explained in When a source is read.
A new source in a ticked collection is read by itself. In these cases you read again yourself:
| What you did | What you do |
|---|---|
| created the table set over collections that already held sources, changed the pipeline, ticked or unticked a collection, or changed a filter | the screen shows the Read again notice with a button |
| removed the cause of failed sources | Retry failed above the list of sources |
Read again reads every source in scope again and removes the tables of sources the table set no longer reads. While a run is under way, a Reading sources bar counts along by itself. Run history shows the runs with their progress and trigger.
Find a table¶
Tables lists the tables the table set holds, with the Source each comes from, its Origin (Worksheet, CSV, Records or In the document), its Columns, its number of Rows and the moment it last changed. Search by the name of the table or the title of its source. Open a table to see it on a page of its own.
A source's drawer in its collection lists its tables too, grouped by the table set that holds them, each a link to the table's page.
A table's page¶
At the top are the table's name, its origin, the source it comes from and its number of rows. Back to the table set returns to the list. Below that:
- Ask the table: the question form (below).
- Rows: the rows in the order the document holds them, page by page.
- Columns: each column's name and type: text, whole number, number, yes/no or date.
Ask the table a question¶
A question is a form, not text: you filter the rows, sort them, or calculate over them, optionally per group. Core checks every column and comparison against the table.
- Under Filters, select Add filter and choose a Column, a Comparison and a Value. Add up to 20 filters; a row must meet all of them. For is one of, enter the values separated by a comma.
- Under Calculation, keep none, show the rows, or choose Count, Sum, Average, Smallest or Largest and the column under Of column. Count can count all rows.
- With a calculation, choose a column under Per group of column to calculate per value of that column, such as the sum per supplier.
- Under Sort by, keep the order of the document or choose a column; with groups you can sort by the result. Choose the Direction.
- Set Most rows and select Ask.
The answer shows the matching rows, or one row per group with the result. When it says the answer is partial, there were more groups than Most rows: make the question narrower. A question that takes too long is refused with a request to add a filter. Asking a question here does not call a model and is not metered as usage.
A model does not ask a table here but through a search tool whose pipeline
has the table set as a source. The console's pipeline editor does not offer a table set as a
source block; make that search tool through the API, from the retrieval/table-set template
(Let a model search).
Delete a table set¶
Select Delete under Delete table set. This deletes the table set with all its tables and rows, and cannot be undone. The collections and their sources stay. A search tool that reads the table set then fails to find it; update such search tools before you delete it.
Labels¶
Use Labels to classify table sets. Rules about Table sets in a key's policy decide by name or by label which table sets that key, and a person, reaches (Rules about a person). The default rules of a key registered without a policy permit every table set.
A key's or an organisation's policy stored before table sets existed, including the default rules
written out when a key was registered, has no rule about Table sets and reaches none: listing
and reading answer 404 and creating answers 403, until you add a rule that permits Table
sets (Add or change a rule).
Inside a table set, a person sees a table only when she may see the source it comes from. See also Build for people.
Empty lists and access restrictions¶
When the left side is empty, this organisation has no table sets.
If knowledge is disabled, Index and search knowledge sources is off in the organisation's ceiling. The administrator of the organisation above turns it on under Permitted capabilities on Settings; for an organisation with none above it, your operator does.