DataTable Filtering
As discussed in the interactivity chapter,
DataTable
includes filtering capabilities.
Set filter_action='native'
for clientside (front-end) filtering
or filter_action='custom'
to perform your own filtering in Python.
filter_action='native'
will work well up to 10,000-100,000 rows.
After which, you may want to use filter_action='custom'
so that your
app sends less data over the network into the browser.
from dash import Dash, dash_table
import datetime
import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminder2007.csv')
df = df[['continent', 'country', 'pop', 'lifeExp']] # prune columns for example
df['Mock Date'] = [
datetime.datetime(2020, 1, 1, 0, 0, 0) + i * datetime.timedelta(hours=13)
for i in range(len(df))
]
app = Dash()
app.layout = dash_table.DataTable(
columns=[
{'name': 'Continent', 'id': 'continent', 'type': 'numeric'},
{'name': 'Country', 'id': 'country', 'type': 'text'},
{'name': 'Population', 'id': 'pop', 'type': 'numeric'},
{'name': 'Life Expectancy', 'id': 'lifeExp', 'type': 'numeric'},
{'name': 'Mock Dates', 'id': 'Mock Date', 'type': 'datetime'}
],
data=df.to_dict('records'),
filter_action='native',
style_table={
'height': 400,
},
style_data={
'width': '150px', 'minWidth': '150px', 'maxWidth': '150px',
'overflow': 'hidden',
'textOverflow': 'ellipsis',
}
)
if __name__ == '__main__':
app.run(debug=True)
Continent | Country | Population | Life Expectancy | Mock Dates |
---|---|---|---|---|
Notes:
- As above, we recommend fixing column widths with filtering. Otherwise, the column widths will grow or shrink depending on how wide the data is within the columns.
- There is a bug with
fixed_rows
that prevents horizontal scroll when no filter results are returned. Until this bug is fixed, we recommend avoidingfixed_rows
. For updates, see plotly/dash-table#746 - The default filtering behavior will depend on the data type of the column (see below). Data types are not inferred, so you have to set them manually.
The example below determines the datatype of the column automatically with Pandas:
from dash import Dash, dash_table
import datetime
import sys
import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminder2007.csv')
df = df[['continent', 'country', 'pop', 'lifeExp']] # prune columns for example
df['Mock Date'] = [
datetime.datetime(2020, 1, 1, 0, 0, 0) + i * datetime.timedelta(hours=13)
for i in range(len(df))
]
app = Dash()
def table_type(df_column):
# Note - this only works with Pandas >= 1.0.0
if sys.version_info < (3, 0): # Pandas 1.0.0 does not support Python 2
return 'any'
if isinstance(df_column.dtype, pd.DatetimeTZDtype):
return 'datetime',
elif (isinstance(df_column.dtype, pd.StringDtype) or
isinstance(df_column.dtype, pd.BooleanDtype) or
isinstance(df_column.dtype, pd.CategoricalDtype) or
isinstance(df_column.dtype, pd.PeriodDtype)):
return 'text'
elif (isinstance(df_column.dtype, pd.SparseDtype) or
isinstance(df_column.dtype, pd.IntervalDtype) or
isinstance(df_column.dtype, pd.Int8Dtype) or
isinstance(df_column.dtype, pd.Int16Dtype) or
isinstance(df_column.dtype, pd.Int32Dtype) or
isinstance(df_column.dtype, pd.Int64Dtype)):
return 'numeric'
else:
return 'any'
app.layout = dash_table.DataTable(
columns=[
{'name': i, 'id': i, 'type': table_type(df[i])} for i in df.columns
],
data=df.to_dict('records'),
filter_action='native',
css=[{
'selector': 'table',
'rule': 'table-layout: fixed' # note - this does not work with fixed_rows
}],
style_table={'height': 400},
style_data={
'width': '{}%'.format(100. / len(df.columns)),
'textOverflow': 'hidden'
}
)
if __name__ == '__main__':
app.run(debug=True)
continent | country | pop | lifeExp | Mock Date |
---|---|---|---|---|
Asia | Afghanistan | 31889923 | 43.828 | 2020-01-01T00:00:00 |
Europe | Albania | 3600523 | 76.423 | 2020-01-01T13:00:00 |
Africa | Algeria | 33333216 | 72.301 | 2020-01-02T02:00:00 |
Africa | Angola | 12420476 | 42.731 | 2020-01-02T15:00:00 |
Americas | Argentina | 40301927 | 75.32 | 2020-01-03T04:00:00 |
Oceania | Australia | 20434176 | 81.235 | 2020-01-03T17:00:00 |
Europe | Austria | 8199783 | 79.829 | 2020-01-04T06:00:00 |
Asia | Bahrain | 708573 | 75.635 | 2020-01-04T19:00:00 |
Asia | Bangladesh | 150448339 | 64.062 | 2020-01-05T08:00:00 |
Europe | Belgium | 10392226 | 79.441 | 2020-01-05T21:00:00 |
Africa | Benin | 8078314 | 56.728 | 2020-01-06T10:00:00 |
Americas | Bolivia | 9119152 | 65.554 | 2020-01-06T23:00:00 |
Europe | Bosnia and Herzegovina | 4552198 | 74.852 | 2020-01-07T12:00:00 |
Africa | Botswana | 1639131 | 50.728 | 2020-01-08T01:00:00 |
Americas | Brazil | 190010647 | 72.39 | 2020-01-08T14:00:00 |
Europe | Bulgaria | 7322858 | 73.005 | 2020-01-09T03:00:00 |
Africa | Burkina Faso | 14326203 | 52.295 | 2020-01-09T16:00:00 |
Africa | Burundi | 8390505 | 49.58 | 2020-01-10T05:00:00 |
Asia | Cambodia | 14131858 | 59.723 | 2020-01-10T18:00:00 |
Africa | Cameroon | 17696293 | 50.43 | 2020-01-11T07:00:00 |
Americas | Canada | 33390141 | 80.653 | 2020-01-11T20:00:00 |
Africa | Central African Republic | 4369038 | 44.74100000000001 | 2020-01-12T09:00:00 |
Africa | Chad | 10238807 | 50.651 | 2020-01-12T22:00:00 |
Americas | Chile | 16284741 | 78.553 | 2020-01-13T11:00:00 |
Asia | China | 1318683096 | 72.961 | 2020-01-14T00:00:00 |
Americas | Colombia | 44227550 | 72.889 | 2020-01-14T13:00:00 |
Africa | Comoros | 710960 | 65.152 | 2020-01-15T02:00:00 |
Africa | Congo, Dem. Rep. | 64606759 | 46.462 | 2020-01-15T15:00:00 |
Africa | Congo, Rep. | 3800610 | 55.322 | 2020-01-16T04:00:00 |
Americas | Costa Rica | 4133884 | 78.782 | 2020-01-16T17:00:00 |
Africa | Cote d'Ivoire | 18013409 | 48.328 | 2020-01-17T06:00:00 |
Europe | Croatia | 4493312 | 75.748 | 2020-01-17T19:00:00 |
Americas | Cuba | 11416987 | 78.273 | 2020-01-18T08:00:00 |
Europe | Czech Republic | 10228744 | 76.486 | 2020-01-18T21:00:00 |
Europe | Denmark | 5468120 | 78.332 | 2020-01-19T10:00:00 |
Africa | Djibouti | 496374 | 54.791 | 2020-01-19T23:00:00 |
Americas | Dominican Republic | 9319622 | 72.235 | 2020-01-20T12:00:00 |
Americas | Ecuador | 13755680 | 74.994 | 2020-01-21T01:00:00 |
Africa | Egypt | 80264543 | 71.33800000000001 | 2020-01-21T14:00:00 |
Americas | El Salvador | 6939688 | 71.878 | 2020-01-22T03:00:00 |
Africa | Equatorial Guinea | 551201 | 51.57899999999999 | 2020-01-22T16:00:00 |
Africa | Eritrea | 4906585 | 58.04 | 2020-01-23T05:00:00 |
Africa | Ethiopia | 76511887 | 52.947 | 2020-01-23T18:00:00 |
Europe | Finland | 5238460 | 79.313 | 2020-01-24T07:00:00 |
Europe | France | 61083916 | 80.657 | 2020-01-24T20:00:00 |
Africa | Gabon | 1454867 | 56.735 | 2020-01-25T09:00:00 |
Africa | Gambia | 1688359 | 59.448 | 2020-01-25T22:00:00 |
Europe | Germany | 82400996 | 79.406 | 2020-01-26T11:00:00 |
Africa | Ghana | 22873338 | 60.022 | 2020-01-27T00:00:00 |
Europe | Greece | 10706290 | 79.483 | 2020-01-27T13:00:00 |
Americas | Guatemala | 12572928 | 70.259 | 2020-01-28T02:00:00 |
Africa | Guinea | 9947814 | 56.007 | 2020-01-28T15:00:00 |
Africa | Guinea-Bissau | 1472041 | 46.388000000000005 | 2020-01-29T04:00:00 |
Americas | Haiti | 8502814 | 60.916 | 2020-01-29T17:00:00 |
Americas | Honduras | 7483763 | 70.19800000000001 | 2020-01-30T06:00:00 |
Asia | Hong Kong, China | 6980412 | 82.208 | 2020-01-30T19:00:00 |
Europe | Hungary | 9956108 | 73.33800000000001 | 2020-01-31T08:00:00 |
Europe | Iceland | 301931 | 81.757 | 2020-01-31T21:00:00 |
Asia | India | 1110396331 | 64.69800000000001 | 2020-02-01T10:00:00 |
Asia | Indonesia | 223547000 | 70.65 | 2020-02-01T23:00:00 |
Asia | Iran | 69453570 | 70.964 | 2020-02-02T12:00:00 |
Asia | Iraq | 27499638 | 59.545 | 2020-02-03T01:00:00 |
Europe | Ireland | 4109086 | 78.885 | 2020-02-03T14:00:00 |
Asia | Israel | 6426679 | 80.745 | 2020-02-04T03:00:00 |
Europe | Italy | 58147733 | 80.546 | 2020-02-04T16:00:00 |
Americas | Jamaica | 2780132 | 72.567 | 2020-02-05T05:00:00 |
Asia | Japan | 127467972 | 82.603 | 2020-02-05T18:00:00 |
Asia | Jordan | 6053193 | 72.535 | 2020-02-06T07:00:00 |
Africa | Kenya | 35610177 | 54.11 | 2020-02-06T20:00:00 |
Asia | Korea, Dem. Rep. | 23301725 | 67.297 | 2020-02-07T09:00:00 |
Asia | Korea, Rep. | 49044790 | 78.623 | 2020-02-07T22:00:00 |
Asia | Kuwait | 2505559 | 77.58800000000001 | 2020-02-08T11:00:00 |
Asia | Lebanon | 3921278 | 71.993 | 2020-02-09T00:00:00 |
Africa | Lesotho | 2012649 | 42.592 | 2020-02-09T13:00:00 |
Africa | Liberia | 3193942 | 45.678 | 2020-02-10T02:00:00 |
Africa | Libya | 6036914 | 73.952 | 2020-02-10T15:00:00 |
Africa | Madagascar | 19167654 | 59.443000000000005 | 2020-02-11T04:00:00 |
Africa | Malawi | 13327079 | 48.303 | 2020-02-11T17:00:00 |
Asia | Malaysia | 24821286 | 74.241 | 2020-02-12T06:00:00 |
Africa | Mali | 12031795 | 54.467 | 2020-02-12T19:00:00 |
Africa | Mauritania | 3270065 | 64.164 | 2020-02-13T08:00:00 |
Africa | Mauritius | 1250882 | 72.801 | 2020-02-13T21:00:00 |
Americas | Mexico | 108700891 | 76.195 | 2020-02-14T10:00:00 |
Asia | Mongolia | 2874127 | 66.803 | 2020-02-14T23:00:00 |
Europe | Montenegro | 684736 | 74.543 | 2020-02-15T12:00:00 |
Africa | Morocco | 33757175 | 71.164 | 2020-02-16T01:00:00 |
Africa | Mozambique | 19951656 | 42.082 | 2020-02-16T14:00:00 |
Asia | Myanmar | 47761980 | 62.069 | 2020-02-17T03:00:00 |
Africa | Namibia | 2055080 | 52.906000000000006 | 2020-02-17T16:00:00 |
Asia | Nepal | 28901790 | 63.785 | 2020-02-18T05:00:00 |
Europe | Netherlands | 16570613 | 79.762 | 2020-02-18T18:00:00 |
Oceania | New Zealand | 4115771 | 80.204 | 2020-02-19T07:00:00 |
Americas | Nicaragua | 5675356 | 72.899 | 2020-02-19T20:00:00 |
Africa | Niger | 12894865 | 56.867 | 2020-02-20T09:00:00 |
Africa | Nigeria | 135031164 | 46.859 | 2020-02-20T22:00:00 |
Europe | Norway | 4627926 | 80.196 | 2020-02-21T11:00:00 |
Asia | Oman | 3204897 | 75.64 | 2020-02-22T00:00:00 |
Asia | Pakistan | 169270617 | 65.483 | 2020-02-22T13:00:00 |
Americas | Panama | 3242173 | 75.53699999999999 | 2020-02-23T02:00:00 |
Americas | Paraguay | 6667147 | 71.752 | 2020-02-23T15:00:00 |
Americas | Peru | 28674757 | 71.421 | 2020-02-24T04:00:00 |
Asia | Philippines | 91077287 | 71.688 | 2020-02-24T17:00:00 |
Europe | Poland | 38518241 | 75.563 | 2020-02-25T06:00:00 |
Europe | Portugal | 10642836 | 78.098 | 2020-02-25T19:00:00 |
Americas | Puerto Rico | 3942491 | 78.74600000000001 | 2020-02-26T08:00:00 |
Africa | Reunion | 798094 | 76.442 | 2020-02-26T21:00:00 |
Europe | Romania | 22276056 | 72.476 | 2020-02-27T10:00:00 |
Africa | Rwanda | 8860588 | 46.242 | 2020-02-27T23:00:00 |
Africa | Sao Tome and Principe | 199579 | 65.528 | 2020-02-28T12:00:00 |
Asia | Saudi Arabia | 27601038 | 72.777 | 2020-02-29T01:00:00 |
Africa | Senegal | 12267493 | 63.062 | 2020-02-29T14:00:00 |
Europe | Serbia | 10150265 | 74.002 | 2020-03-01T03:00:00 |
Africa | Sierra Leone | 6144562 | 42.568000000000005 | 2020-03-01T16:00:00 |
Asia | Singapore | 4553009 | 79.972 | 2020-03-02T05:00:00 |
Europe | Slovak Republic | 5447502 | 74.663 | 2020-03-02T18:00:00 |
Europe | Slovenia | 2009245 | 77.926 | 2020-03-03T07:00:00 |
Africa | Somalia | 9118773 | 48.159 | 2020-03-03T20:00:00 |
Africa | South Africa | 43997828 | 49.339 | 2020-03-04T09:00:00 |
Europe | Spain | 40448191 | 80.941 | 2020-03-04T22:00:00 |
Asia | Sri Lanka | 20378239 | 72.396 | 2020-03-05T11:00:00 |
Africa | Sudan | 42292929 | 58.556 | 2020-03-06T00:00:00 |
Africa | Swaziland | 1133066 | 39.613 | 2020-03-06T13:00:00 |
Europe | Sweden | 9031088 | 80.884 | 2020-03-07T02:00:00 |
Europe | Switzerland | 7554661 | 81.70100000000001 | 2020-03-07T15:00:00 |
Asia | Syria | 19314747 | 74.143 | 2020-03-08T04:00:00 |
Asia | Taiwan | 23174294 | 78.4 | 2020-03-08T17:00:00 |
Africa | Tanzania | 38139640 | 52.517 | 2020-03-09T06:00:00 |
Asia | Thailand | 65068149 | 70.616 | 2020-03-09T19:00:00 |
Africa | Togo | 5701579 | 58.42 | 2020-03-10T08:00:00 |
Americas | Trinidad and Tobago | 1056608 | 69.819 | 2020-03-10T21:00:00 |
Africa | Tunisia | 10276158 | 73.923 | 2020-03-11T10:00:00 |
Europe | Turkey | 71158647 | 71.777 | 2020-03-11T23:00:00 |
Africa | Uganda | 29170398 | 51.542 | 2020-03-12T12:00:00 |
Europe | United Kingdom | 60776238 | 79.425 | 2020-03-13T01:00:00 |
Americas | United States | 301139947 | 78.242 | 2020-03-13T14:00:00 |
Americas | Uruguay | 3447496 | 76.384 | 2020-03-14T03:00:00 |
Americas | Venezuela | 26084662 | 73.747 | 2020-03-14T16:00:00 |
Asia | Vietnam | 85262356 | 74.249 | 2020-03-15T05:00:00 |
Asia | West Bank and Gaza | 4018332 | 73.422 | 2020-03-15T18:00:00 |
Asia | Yemen, Rep. | 22211743 | 62.698 | 2020-03-16T07:00:00 |
Africa | Zambia | 11746035 | 42.38399999999999 | 2020-03-16T20:00:00 |
Africa | Zimbabwe | 12311143 | 43.487 | 2020-03-17T09:00:00 |
Filtering Operators
The filtering syntax is data-type specific. Data types are not inferred, they must be set manually. If a type is not specified, then we assume it is a string (text).
Text & String Filtering
United
= United
United States
"United States"
= United States
= "United States"
contains United
> United
>= United
< United
<= United
By default, the columns with the "text" type use the
contains
operator. So, searching United
is the same as
contains United
For legacy purposes, eq
can also be substituted for =
.
>
, >=
, <
, and <=
compare strings in dictionary order,
with numbers and most symbols coming before letters,
and uppercase coming before lowercase.
If you have quotes in the string, you can use a different quote, or
escape the quote character. So eq 'Say "Yes!"'
and
="Say \\"Yes!\\""
are the same.
Numeric Filtering
43.828
= 43.828
> 43.828
>= 43.828
< 43.828
<= 43.828
By default, columns with the numeric
type use the =
operator.
So, searching 43.828
is the same as = 43.828
.
Datetime Filtering
2020
2020-01
2020-01-01
2020-01-01 04:01
2020-01-01 04:01:10
datestartswith 2020
datestartswith 2020-01
datestartswith 2020-01-01
datestartswith 2020-01-01 04:01
datestartswith 2020-01-01 04:01:10
> 2020-01
> 2020-01-20
>= 2020-01
>= 2020-01-20
< 2020-01
< 2020-01-20
<= 2020-01
<= 2020-01-20
Operators
Many operators have two forms: a symbol (=
) and a word (eq
) that
can be used interchangeably.
number columns |
Are the two numbers equal? Regardless of type, will first try to convert both sides to numbers and compare the numbers. If either cannot be converted to a number, looks for an exact match. |
text and any columns |
Does the text value contain the requested substring? May match the beginning, end, or anywhere in the middle. The match is case-sensitive and exact. |
datetime columns' |
Does the datetime start with the given parts? Enter a partial
datetime, this will match any date that has at least as much
precision and starts with the same pieces. For example,
datestartswith '2018-03-01' will match '2018-03-01 12:59' but
not '2018-03' even though we interpret '2018-03-01' and
'2018-03' both to mean the first instant of March, 2018. |
|
Comparison: greater than, less than, greater or equal, less or equal, and not equal. Two strings compare by their dictionary order, with numbers and most symbols coming before letters, and uppercase coming before lowercase. |
Back-end Filtering
For large dataframes, you can perform the filtering in Python instead of the default clientside filtering. You can find more information on performing operations in python in the Python Callbacks chapter.
The syntax is (now) the same as front-end filtering, but it's up to the developer to implement the logic to apply these filters on the Python side. In the future we may accept any filter strings, to allow you to write your own expression query language.
Example:
from dash import Dash, dash_table, Input, Output, callback
import pandas as pd
app = Dash()
df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminder2007.csv')
app.layout = dash_table.DataTable(
id='table-filtering-be',
columns=[
{"name": i, "id": i} for i in sorted(df.columns)
],
filter_action='custom',
filter_query=''
)
operators = [['ge ', '>='],
['le ', '<='],
['lt ', '<'],
['gt ', '>'],
['ne ', '!='],
['eq ', '='],
['contains '],
['datestartswith ']]
def split_filter_part(filter_part):
for operator_type in operators:
for operator in operator_type:
if operator in filter_part:
name_part, value_part = filter_part.split(operator, 1)
name = name_part[name_part.find('{') + 1: name_part.rfind('}')]
value_part = value_part.strip()
v0 = value_part[0]
if (v0 == value_part[-1] and v0 in ("'", '"', '`')):
value = value_part[1: -1].replace('\\' + v0, v0)
else:
try:
value = float(value_part)
except ValueError:
value = value_part
# word operators need spaces after them in the filter string,
# but we don't want these later
return name, operator_type[0].strip(), value
return [None] * 3
@callback(
Output('table-filtering-be', "data"),
Input('table-filtering-be', "filter_query"))
def update_table(filter):
filtering_expressions = filter.split(' && ')
dff = df
for filter_part in filtering_expressions:
col_name, operator, filter_value = split_filter_part(filter_part)
if operator in ('eq', 'ne', 'lt', 'le', 'gt', 'ge'):
# these operators match pandas series operator method names
dff = dff.loc[getattr(dff[col_name], operator)(filter_value)]
elif operator == 'contains':
dff = dff.loc[dff[col_name].str.contains(filter_value)]
elif operator == 'datestartswith':
# this is a simplification of the front-end filtering logic,
# only works with complete fields in standard format
dff = dff.loc[dff[col_name].str.startswith(filter_value)]
return dff.to_dict('records')
if __name__ == '__main__':
app.run(debug=True)
continent | country | gdpPercap | lifeExp | pop |
---|---|---|---|---|
Advanced filter usage
Filter queries can be as simple or as complicated as you want them to be. When something is typed into a column filter, it is automatically converted to a filter query on that column only.
from dash import Dash, dcc, html, Input, Output, dash_table, callback
import pandas as pd
import json
df = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/gapminder2007.csv')
df['id'] = df['country']
df.set_index('id', inplace=True, drop=False)
app = Dash()
app.layout = html.Div([
dcc.RadioItems(
[{'label': 'Read filter_query', 'value': 'read'}, {'label': 'Write to filter_query', 'value': 'write'}],
'read',
id='filter-query-read-write',
),
html.Br(),
dcc.Input(id='filter-query-input', placeholder='Enter filter query'),
html.Div(id='filter-query-output'),
html.Hr(),
dash_table.DataTable(
id='datatable-advanced-filtering',
columns=[
{'name': i, 'id': i, 'deletable': True} for i in df.columns
# omit the id column
if i != 'id'
],
data=df.to_dict('records'),
editable=True,
page_action='native',
page_size=10,
filter_action="native"
),
html.Hr(),
html.Div(id='datatable-query-structure', style={'whitespace': 'pre'})
])
@callback(
Output('filter-query-input', 'style'),
Output('filter-query-output', 'style'),
Input('filter-query-read-write', 'value')
)
def query_input_output(val):
input_style = {'width': '100%'}
output_style = {}
if val == 'read':
input_style.update(display='none')
output_style.update(display='inline-block')
else:
input_style.update(display='inline-block')
output_style.update(display='none')
return input_style, output_style
@callback(
Output('datatable-advanced-filtering', 'filter_query'),
Input('filter-query-input', 'value')
)
def write_query(query):
if query is None:
return ''
return query
@callback(
Output('filter-query-output', 'children'),
Input('datatable-advanced-filtering', 'filter_query')
)
def read_query(query):
if query is None:
return "No filter query"
return dcc.Markdown('`filter_query = "{}"`'.format(query))
@callback(
Output('datatable-query-structure', 'children'),
Input('datatable-advanced-filtering', 'derived_filter_query_structure')
)
def display_query(query):
if query is None:
return ''
return html.Details([
html.Summary('Derived filter query structure'),
html.Div(dcc.Markdown('''```json
{}
```'''.format(json.dumps(query, indent=4))))
])
if __name__ == '__main__':
app.run(debug=True)
country | pop | continent | lifeExp | gdpPercap |
---|---|---|---|---|
Afghanistan | 31889923 | Asia | 43.828 | 974.5803384 |
Albania | 3600523 | Europe | 76.423 | 5937.029525999999 |
Algeria | 33333216 | Africa | 72.301 | 6223.367465 |
Angola | 12420476 | Africa | 42.731 | 4797.231267 |
Argentina | 40301927 | Americas | 75.32 | 12779.37964 |
Australia | 20434176 | Oceania | 81.235 | 34435.367439999995 |
Austria | 8199783 | Europe | 79.829 | 36126.4927 |
Bahrain | 708573 | Asia | 75.635 | 29796.04834 |
Bangladesh | 150448339 | Asia | 64.062 | 1391.253792 |
Belgium | 10392226 | Europe | 79.441 | 33692.60508 |
The filter_query
property is written to when the user
filters the data by using the column filters. For example, if
a user types ge 100000000
in the pop
column filter, and
Asia
in the continent
column filter, filter_query
will
look like this:
{pop} ge 100000000 && {continent} contains "Asia"
Try typing those values into the column filters in the app above, and ensure that the "Read filter_query" option is selected.
The filter_query
property can also be written to. This might
be useful when performing more complex filtering,
like if you want to filter a column based on two (or more)
conditions. For instance, say that we want countries with a
population greater than 100 million, but less than 500
million. Then our filter_query
would be as follows:
{pop} ge 100000000 and {pop} le 500000000
Select the "Write to filter_query" option in the app above, and try it out by copying and pasting the filter query above into the input box.
Say that we now want to get a bit more advanced, and cross-filter between columns; for instance, we only want the results that are located in Asia. Now, our filter query becomes:
{pop} ge 100000000 and {pop} le 500000000 and {continent} eq "Asia"
We can make the expression even more complex. For example, let's say we want all of those countries with the populations that fall within our boundaries and that are in Asia, but for some reason we also want to include Singapore. This results in a filter query that is a little more long-winded:
(({pop} ge 100000000 and {pop} le 500000000) or {country} eq "Singapore") and {continent} eq "Asia"
Note that we've grouped expressions together using parentheses. This is part of the filtering syntax. Just as is true in mathematical expressions, the expressions in the innermost parentheses are evaluated first.
Symbol-based versus letter-based operators
An important thing to notice is that the two types of
relational operators that can be used in the column filters
(symbol-based, like >=
, and letter-based, like ge
) are not
converted into one another when filter_query
is being
constructed from the values in the column filters. Therefore,
if using filter_query
to implement backend filtering, it's
necessary to take both of these forms of the
"greater-than-or-equal-to" operator into account when parsing
the query string (or ensure that the user only uses the ones
that the backend can parse).
However, in the case of the logical operator and/&&
, when
the table is constructing the query string, the symbol-based
representation will always be used.
Derived filter query structure
The derived_filter_query_structure
prop is a dictionary
representation of the query syntax tree. You can use the value
of this property to implement backend filtering.
For a query that describes a relationship between two values, there are three components: the operation, the left-hand side, and the right-hand side. For instance, take the following query:
{pop} ge 100000000
The operation here is ge
(i.e., >=
), the left-hand side is
the field pop
(corresponding to the column pop
), and the
right-hand side is the value 100000000
. As the queries
become increasingly complex, so do the query structures. Try
it out by expanding the "Derived filter query structure" in
the example app above.
Note that for all operators, there are two keys subType
and
value
that correspond to, respectively, the symbol-based
representation and the originally inputted representation of
the operator. So, in the case of the query above, subType
will be >=
and value
will be ge
; if our query string
were {pop} >= 100000000
instead, both subType
and value
will be >=
.
Backend filtering with pandas
and derived_filter_query_structure
It's likely that your data are already in a pandas
dataframe. Using the derived_filter_query_structure
in
conjunction with pandas
filters can enable you to do some
pretty heavy lifting with the table! You can see an example of
this below.
from dash import Dash, Input, Output, callback
from dash.dash_table import DataTable
import json
import pandas as pd
types = {
'id': 'numeric',
'Complaint_ID': 'numeric',
'ZIP_code': 'numeric',
'Date_received': 'datetime',
'Date_sent_to_company': 'datetime',
}
df = pd.read_csv('https://github.com/plotly/datasets/raw/master/26k-consumer-complaints.csv')
df['id'] = df['Unnamed: 0']
df = df.drop(['Unnamed: 0'], axis=1)
df = df.reindex(columns=['id']+df.columns[:-1].tolist())
df.columns = [column.replace(" ", "_") for column in df.columns]
df.columns = [column.replace("-", "_") for column in df.columns]
app = Dash()
app.scripts.config.serve_locally = True
app.layout = DataTable(
id='demo-table',
data=df.to_dict('records'),
columns=[{ 'id': i, 'name': i, 'type': types.get(i, 'any')} for i in df.columns],
filter_action='custom',
page_action='native',
page_size=15,
virtualization=True,
style_cell={
'min-width': '100px'
},
css=[
{ 'selector': '.row-1', 'rule': 'min-height: 500px;' }
]
)
def to_string(filter):
operator_type = filter.get('type')
operator_subtype = filter.get('subType')
if operator_type == 'relational-operator':
if operator_subtype == '=':
return '=='
else:
return operator_subtype
elif operator_type == 'logical-operator':
if operator_subtype == '&&':
return '&'
else:
return '|'
elif operator_type == 'expression' and operator_subtype == 'value' and type(filter.get('value')) == str:
return '"{}"'.format(filter.get('value'))
else:
return filter.get('value')
def construct_filter(derived_query_structure, df, complexOperator=None):
# there is no query; return an empty filter string and the
# original dataframe
if derived_query_structure is None:
return ('', df)
# the operator typed in by the user; can be both word-based or
# symbol-based
operator_type = derived_query_structure.get('type')
# the symbol-based representation of the operator
operator_subtype = derived_query_structure.get('subType')
# the LHS and RHS of the query, which are both queries themselves
left = derived_query_structure.get('left', None)
right = derived_query_structure.get('right', None)
# the base case
if left is None and right is None:
return (to_string(derived_query_structure), df)
# recursively apply the filter on the LHS of the query to the
# dataframe to generate a new dataframe
(left_query, left_df) = construct_filter(left, df)
# apply the filter on the RHS of the query to this new dataframe
(right_query, right_df) = construct_filter(right, left_df)
# 'datestartswith' and 'contains' can't be used within a pandas
# filter string, so we have to do this filtering ourselves
if complexOperator is not None:
right_query = right.get('value')
# perform the filtering to generate a new dataframe
if complexOperator == 'datestartswith':
return ('', right_df[right_df[left_query].astype(str).str.startswith(right_query)])
elif complexOperator == 'contains':
return ('', right_df[right_df[left_query].astype(str).str.contains(right_query)])
if operator_type == 'relational-operator' and operator_subtype in ['contains', 'datestartswith']:
return construct_filter(derived_query_structure, df, complexOperator=operator_subtype)
# construct the query string; return it and the filtered dataframe
return ('{} {} {}'.format(
left_query,
to_string(derived_query_structure) if left_query != '' and right_query != '' else '',
right_query
).strip(), right_df)
@callback(
Output("demo-table", "data"),
Input("demo-table", "derived_filter_query_structure")
)
def onFilterUpdate(derived_query_structure):
(pd_query_string, df_filtered) = construct_filter(derived_query_structure, df)
if pd_query_string != '':
df_filtered = df_filtered.query(pd_query_string)
return df_filtered.to_dict('records')
if __name__ == "__main__":
app.run(debug=True)
id | Complaint_ID | Product | Sub_product | Issue | Sub_issue | State | ZIP_code | Date_received | Date_sent_to_company | Company | Company_response | Timely_response? | Consumer_disputed? |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 1291006 | Debt collection | Communication tactics | Frequent or repeated calls | TX | 76119 | 2015-03-19 | 2015-03-19 | Premium Asset Services, LLC | In progress | Yes | ||
1 | 1290580 | Debt collection | Medical | Cont'd attempts collect debt not owed | Debt is not mine | TX | 77479 | 2015-03-19 | 2015-03-19 | Accounts Receivable Consultants Inc. | Closed with explanation | Yes | |
2 | 1290564 | Mortgage | FHA mortgage | Application, originator, mortgage broker | MA | 2127 | 2015-03-19 | 2015-03-19 | RBS Citizens | Closed with explanation | Yes | Yes | |
3 | 1291615 | Credit card | Other | CA | 92592 | 2015-03-19 | 2015-03-19 | Navy FCU | In progress | Yes | |||
4 | 1292165 | Debt collection | Non-federal student loan | Cont'd attempts collect debt not owed | Debt resulted from identity theft | 43068 | 2015-03-19 | 2015-03-19 | Transworld Systems Inc. | In progress | Yes | ||
5 | 1291176 | Debt collection | Payday loan | Communication tactics | Called after sent written cease of comm | OH | 43068 | 2015-03-19 | 2015-03-19 | ACE Cash Express Inc. | In progress | Yes | |
6 | 1288848 | Consumer loan | Installment loan | Managing the loan or lease | OH | 44241 | 2015-03-18 | 2015-03-18 | CashCall, Inc. | Closed with explanation | Yes | Yes | |
7 | 1288788 | Debt collection | Payday loan | Communication tactics | Called after sent written cease of comm | CA | 95124 | 2015-03-18 | 2015-03-18 | ACE Cash Express Inc. | Closed with explanation | Yes | |
8 | 1288324 | Debt collection | Other (phone, health club, etc.) | Cont'd attempts collect debt not owed | Debt was paid | NJ | 7067 | 2015-03-18 | 2015-03-18 | Credit Protection Association, L.P. | Closed with non-monetary relief | Yes | |
9 | 1288304 | Debt collection | Payday loan | Taking/threatening an illegal action | Threatened arrest/jail if do not pay | TX | 77433 | 2015-03-18 | 2015-03-18 | Cottonwood Financial Ltd. | Closed with explanation | Yes | Yes |
10 | 1289772 | Debt collection | Other (phone, health club, etc.) | False statements or representation | Impersonated an attorney or official | ND | 58237 | 2015-03-18 | 2015-03-18 | Niagara Portfolio Solutions LLC | Closed with explanation | Yes | |
11 | 1288903 | Bank account or service | Other bank product/service | Deposits and withdrawals | RI | 2893 | 2015-03-18 | 2015-03-18 | Santander Bank US | In progress | Yes | ||
12 | 1289633 | Debt collection | Other (phone, health club, etc.) | Cont'd attempts collect debt not owed | Debt is not mine | CO | 80301 | 2015-03-18 | 2015-03-18 | FAIR COLLECTIONS & OUTSOURCING, INC. | Closed with explanation | Yes | |
13 | 1289273 | Payday loan | Can't contact lender | TX | 78205 | 2015-03-18 | 2015-03-18 | ACE Cash Express Inc. | In progress | Yes |