I suggest that you read the decoding of the 2017 report by Igor Strykhar "ClickHouse - Visually Fast and Visual Data Analysis in Tabix".
Web interface for ClickHouse in a Tabix project.
Key features:
- Works with ClickHouse directly from the browser, without the need to install additional software;
- Syntax highlighted query editor
- Autocompletion of teams;
- Graphical analysis tools for query execution;
- Color schemes to choose from.
I am the technical director of the media2. We are a news exchange news aggregator. We store a lot of data that we receive from our partners and register it with ClickHouse - about 30,000 requests per second.
These are data such as:
- News Clicks.
- Displays news in the aggregator.
- Displays banners on our network.
- And we register events from our own counter, which is similar to Yandex.Metrica. This is our own microanalysis.
We had a very hectic life before ClickHouse. We were very tormented, trying to store this data somewhere and somehow analyze them.
Life before ClickHouse - infiniDB
The first thing we had was infiniDB. She lived with us for 4 years. We started it with difficulty.
- . .
- . , CSV- - .
- . , . .
- . , .
2016 , ClickHouse.
ClickHouse — Cassandra
. . infiniDB , , - , .
. Cassandra. Cassandra . 10 000 . 2 000 - .
. . , Cassandra. . .
ClickHouse – Druid
, . 2016 Druid.
Druid – , Java. . clickstream, - .
Druid 0.9.X.
. . - . .
, - . OpenSource – Tranquility, . , .
- . , , , , . - , . . . , , . . . , .
. , habr, , ClickHouse. , .
2 ClickHouse. . infiniDB – , Druid – . Cassandra . php Cassandra , .
? . . . . . ClickHouse – , .
, ClickHouse OpenSource, . 2 web, , , . . .
- ClickHouse. , :
, . . , .
, Druid. Druid, SuperSet. . Druid .
ClickHouse . . . , , : SELECT event, GROUP BY event. ClickHouse.
– Apache Zeppelin. . . , , . , - ClickHouse .
ClickHouse, . . , -. , . . .
– Redash.IO. Redash . . . . DataSource. . . ClickHouse, MySQL, PostgreSQL .
( 2017 ) Grafana. Grafana, , - , - ClickHouse . , . . . . - - , , ClickHouse.
. . EventSQL, SeperSet, Zeppelin.
? , , . ClickHouse – . , , . . . . , .
3 . 330 Tabix.
, ClickHouse-Frontend, . Tabix.
?
. SQL ClickHouse. .
Tabix. . – . – .
, .
, , . . ctrl , . Tabix , . . ClickHouse.
, , , . . , . , . . , .
, , , ClickHouse . - , ClickHouse join, . , - , , , . 200-300 , - - .
, ( 13:46 https://youtu.be/w1-XsL3nbRg?t=826)
, . – , . – workspace. . .
. , Tabix, .
Hotkey – ( 14:39 https://youtu.be/w1-XsL3nbRg?t=879)
hotkey . . , .
, . . sin, cos tg. , . . . . - . , - . , - , - .
. , Redmine Markdown. - , . , «Copy to Redmine» Redmine Markdown Where.
– . «date». ClickHouse - , , . . . , , . . , . . . , , - .
Tabix «Stats», , . . , . .
. ClickHouse, - . - .
– . , : sin, cos 0 299. «Draw» sin cos.
, . . . .
.
.
.
. , , , . . , . .
– Treemap.
Sankeys – . Streamgrahps, River. River. - . . .
– . , , , , , , , . , , .
, , .
Google map. , , Google map, .
, Tabix .
– ClickHouse. «», . , «referrer» - 730 Gb. , 700 GB, . . 2 TB, .
«request_id», . , .
.
– . realtime c ClickHouse , . Grafana. , .
– . , . , . , 200 GB . . . 30 GB, . . .
! OpenSource
. , , OpenSource, . .
, ? ?
, . ., , . . , OpenSource. MySQL , , PostgreSQL. . . Tabix ClickHouse, .
, . . , , , . , php , . . . . ? ? .
. . 330 . , , . 3 . Javascript. , , Javascript, . , – , . .
! . Tableau ?
. Tabix , .
?
, .
, *Tableau*? ?
ClickHouse. Tableau, . , Tabix . , CSV BI. - . , , – . 5 000 , 6 000 , , .
. . - , ?
. , 10 000 . ?
, ? , ?
, , . -. -, . Tabix .
. ?
, .
ClickHouse, ClickHouse production-?
, . . . production 3. ETL, . . . , . MongoDB, Cassandra, MySQL. ClickHouse . . 3 . 6 . ClickHouse.
, . . . .?
Google map, .. . , .
– Google map. «DRAW_GMAPS», . «DRAW_YMAPS», . . .. Javascript, . . , ClickHouse Javascript, . , . , . . , , . .
. . ?
. , , , .
! , ClickHouse , . , , ClickHouse, . , , , - ? , – . ?
– , , -. , Druid, roadmap - 50 % – , . , ClickHouse. , , roadmap. , Data Science, . Tabix – . , Zeppelin. . Redash , , . SuperSet , . , , .
, Pull request ?
.
! . – Javascript. Javascript - - ?*
Javascript.
?
Angular.
. . R *Shiny**?*
. .
.
. , , , .
*, , .
, . . : «- . , ». . , , . . . – , R , «R ».
!
. - , ?
CSV, Excel.
, , ? , .
«» « png, jpg».
!
P.S. - tabix
- Unzip, copy the directory
buildto nginx root_path - Configure nginx