🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 192 (from laksa020)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa192.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/\')), getAhrefsUnparsedNoserviceFromURL(\'http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/\'))) FORMAT JSONEachRow'

Response:

{"found_url":"http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/","crawl_time":1774072252,"first_indexed_time":1590141513,"http_code":200,"src_unparsed":"net,devdoc!\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/ h80","src_root_hash":"8504065000381170792","history_drop_reason":null,"meta_title":"SELECT - ClickHouse Documentation","meta_descriptions":[],"attrs_boilerpipe_text":"SELECT Queries Syntax\n¶\nSELECT\nperforms data retrieval.\nSELECT\n[\nDISTINCT\n]\nexpr_list\n[\nFROM\n[\ndb\n.]\ntable\n|\n(\nsubquery\n)\n|\ntable_function\n]\n[\nFINAL\n]\n[\nSAMPLE\nsample_coeff\n]\n[\nARRAY\nJOIN\n...]\n[\nGLOBAL\n]\n[\nANY\n|\nALL\n]\n[\nINNER\n|\nLEFT\n|\nRIGHT\n|\nFULL\n|\nCROSS\n]\n[\nOUTER\n]\nJOIN\n(\nsubquery\n)\n|\ntable\nUSING\ncolumns_list\n[\nPREWHERE\nexpr\n]\n[\nWHERE\nexpr\n]\n[\nGROUP\nBY\nexpr_list\n]\n[\nWITH\nTOTALS\n]\n[\nHAVING\nexpr\n]\n[\nORDER\nBY\nexpr_list\n]\n[\nLIMIT\n[\nn\n,\n]\nm\n]\n[\nUNION\nALL\n...]\n[\nINTO\nOUTFILE\nfilename\n]\n[\nFORMAT\nformat\n]\n[\nLIMIT\nn\nBY\ncolumns\n]\nAll the clauses are optional, except for the required list of expressions immediately after SELECT.\nThe clauses below are described in almost the same order as in the query execution conveyor.\nIf the query omits the\nDISTINCT\n,\nGROUP BY\nand\nORDER BY\nclauses and the\nIN\nand\nJOIN\nsubqueries, the query will be completely stream processed, using O(1) amount of RAM.\nOtherwise, the query might consume a lot of RAM if the appropriate restrictions are not specified:\nmax_memory_usage\n,\nmax_rows_to_group_by\n,\nmax_rows_to_sort\n,\nmax_rows_in_distinct\n,\nmax_bytes_in_distinct\n,\nmax_rows_in_set\n,\nmax_bytes_in_set\n,\nmax_rows_in_join\n,\nmax_bytes_in_join\n,\nmax_bytes_before_external_sort\n,\nmax_bytes_before_external_group_by\n. For more information, see the section \"Settings\". It is possible to use external sorting (saving temporary tables to a disk) and external aggregation.\nThe system does not have \"merge join\"\n.\nFROM Clause\n¶\nIf the FROM clause is omitted, data will be read from the\nsystem.one\ntable.\nThe 'system.one' table contains exactly one row (this table fulfills the same purpose as the DUAL table found in other DBMSs).\nThe FROM clause specifies the table to read data from, or a subquery, or a table function; ARRAY JOIN and the regular JOIN may also be included (see below).\nInstead of a table, the SELECT subquery may be specified in brackets.\nIn this case, the subquery processing pipeline will be built into the processing pipeline of an external query.\nIn contrast to standard SQL, a synonym does not need to be specified after a subquery. For compatibility, it is possible to write 'AS name' after a subquery, but the specified name isn't used anywhere.\nA table function may be specified instead of a table. For more information, see the section \"Table functions\".\nTo execute a query, all the columns listed in the query are extracted from the appropriate table. Any columns not needed for the external query are thrown out of the subqueries.\nIf a query does not list any columns (for example, SELECT count() FROM t), some column is extracted from the table anyway (the smallest one is preferred), in order to calculate the number of rows.\nThe FINAL modifier can be used only for a SELECT from a CollapsingMergeTree table. When you specify FINAL, data is selected fully \"collapsed\". Keep in mind that using FINAL leads to a selection that includes columns related to the primary key, in addition to the columns specified in the SELECT. Additionally, the query will be executed in a single stream, and data will be merged during query execution. This means that when using FINAL, the query is processed more slowly. In most cases, you should avoid using FINAL. For more information, see the section \"CollapsingMergeTree engine\".\nSAMPLE Clause\n¶\nThe\nSAMPLE\nclause allows for approximated query processing. Approximated query processing is only supported by the tables in the\nMergeTree\nfamily, and only if the sampling expression was specified during table creation (see\nMergeTree engine\n).\nThe features of data sampling are listed below:\nData sampling is a deterministic mechanism. The result of the same\nSELECT .. SAMPLE\nquery is always the same.\nSampling works consistently for different tables. For tables with a single sampling key, a sample with the same coefficient always selects the same subset of possible data. For example, a sample of user IDs takes rows with the same subset of all the possible user IDs from different tables. This allows using the sample in subqueries in the\nIN\nclause, as well as for manually correlating results of different queries with samples.\nSampling allows reading less data from a disk. Note that for this you must specify the sampling key correctly. For more details see\nCreating a MergeTree Table\n.\nThe\nSAMPLE\nclause can be specified in several ways:\nSAMPLE k\n, where\nk\nis a decimal number from 0 to 1. The query is executed on\nk\npercent of data. For example,\nSAMPLE 0.1\nruns the query on 10% of data.\nRead more\nSAMPLE n\n, where\nn\nis a sufficiently large integer. The query is executed on a sample of at least\nn\nrows (but not significantly more than this). For example,\nSAMPLE 10000000\nruns the query on a minimum of 10,000,000 rows.\nRead more\nSAMPLE k OFFSET m\nwhere\nk\nand\nm\nare numbers from 0 to 1. The query is executed on a sample of\nk\npercent of the data. The data used for the sample is offset by\nm\npercent.\nRead more\nSAMPLE k\n¶\nIn a\nSAMPLE k\nclause,\nk\nis a percent amount of data that the sample is taken from. The example is shown below:\nSELECT\nTitle\n,\ncount\n()\n*\n10\nAS\nPageViews\nFROM\nhits_distributed\nSAMPLE\n0\n.\n1\nWHERE\nCounterID\n=\n34\nGROUP\nBY\nTitle\nORDER\nBY\nPageViews\nDESC\nLIMIT\n1000\nIn this example, the query is executed on a sample from 0.1 (10%) of data. Values of aggregate functions are not corrected automatically, so to get an approximate result, the value 'count()' is manually multiplied by 10.\nSAMPLE n\n¶\nIn this case, the query is executed on a sample of at least\nn\nrows, where\nn\nis a sufficiently large integer. For example,\nSAMPLE 10000000\n.\nSince the minimum unit for data reading is one granule (its size is set by the\nindex_granularity\nsetting), it makes sense to set a sample that is much larger than the size of the granule.\nWhen using the\nSAMPLE n\nclause, the relative coefficient is calculated dynamically. Since you do not know which relative percent of data was processed, you do not know the coefficient the aggregate functions should be multiplied by (for example, you do not know if the\nSAMPLE 1000000\nwas taken from a set of 10,000,000 rows or from a set of 1,000,000,000 rows). In this case, use the\n_sample_factor\ncolumn to get the approximate result.\nThe\n_sample_factor\nis a virtual column that ClickHouse stores relative coefficients in. This column is created automatically when you create a table with the specified sampling key. The usage example is shown below:\nSELECT\nsum\n(\nDuration\n*\n_sample_factor\n)\nFROM\nvisits\nSAMPLE\n10000000\nIf you need to get the approximate count of rows in a\nSELECT .. SAMPLE n\nquery, get the sum() of\n_sample_factor\ncolumn instead of counting\ncount(column * _sample_factor)\nvalue. For example:\nSELECT\nsum\n(\n_sample_factor\n)\nFROM\nvisits\nSAMPLE\n10000000\nNote that to calculate the average in a\nSELECT .. SAMPLE n\nquery, you do not need to use\n_sample_factor\ncolumn:\nSELECT\navg\n(\nDuration\n)\nFROM\nvisits\nSAMPLE\n10000000\nSAMPLE k OFFSET m\n¶\nYou can specify the\nSAMPLE k OFFSET m\nclause, where\nk\nand\nm\nare numbers from 0 to 1. Examples are shown below.\nExample 1.\nSAMPLE\n1\n\/\n10\nIn this example, the sample is the 1\/10th of all data:\n[++------------------]\nExample 2.\nSAMPLE\n1\n\/\n10\nOFFSET\n1\n\/\n2\nHere, the sample of 10% is taken from the second half of data.\n[----------++--------]\nARRAY JOIN Clause\n¶\nAllows executing JOIN with an array or nested data structure. The intent is similar to the 'arrayJoin' function, but its functionality is broader.\nARRAY JOIN\nis essentially\nINNER JOIN\nwith an array. Example:\n:) CREATE TABLE arrays_test (s String, arr Array(UInt8)) ENGINE = Memory\n\nCREATE TABLE arrays_test\n(\n    s String,\n    arr Array(UInt8)\n) ENGINE = Memory\n\nOk.\n\n0 rows in set. Elapsed: 0.001 sec.\n\n:) INSERT INTO arrays_test VALUES ('Hello', [1,2]), ('World', [3,4,5]), ('Goodbye', [])\n\nINSERT INTO arrays_test VALUES\n\nOk.\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT * FROM arrays_test\n\nSELECT *\nFROM arrays_test\n\n┌─s───────┬─arr─────┐\n│ Hello   │ [1,2]   │\n│ World   │ [3,4,5] │\n│ Goodbye │ []      │\n└─────────┴─────────┘\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT s, arr FROM arrays_test ARRAY JOIN arr\n\nSELECT s, arr\nFROM arrays_test\nARRAY JOIN arr\n\n┌─s─────┬─arr─┐\n│ Hello │   1 │\n│ Hello │   2 │\n│ World │   3 │\n│ World │   4 │\n│ World │   5 │\n└───────┴─────┘\n\n5 rows in set. Elapsed: 0.001 sec.\nAn alias can be specified for an array in the ARRAY JOIN clause. In this case, an array item can be accessed by this alias, but the array itself by the original name. Example:\n:) SELECT s, arr, a FROM arrays_test ARRAY JOIN arr AS a\n\nSELECT s, arr, a\nFROM arrays_test\nARRAY JOIN arr AS a\n\n┌─s─────┬─arr─────┬─a─┐\n│ Hello │ [1,2]   │ 1 │\n│ Hello │ [1,2]   │ 2 │\n│ World │ [3,4,5] │ 3 │\n│ World │ [3,4,5] │ 4 │\n│ World │ [3,4,5] │ 5 │\n└───────┴─────────┴───┘\n\n5 rows in set. Elapsed: 0.001 sec.\nMultiple arrays of the same size can be comma-separated in the ARRAY JOIN clause. In this case, JOIN is performed with them simultaneously (the direct sum, not the direct product). Example:\n:) SELECT s, arr, a, num, mapped FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(x -> x + 1, arr) AS mapped\n\nSELECT s, arr, a, num, mapped\nFROM arrays_test\nARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(lambda(tuple(x), plus(x, 1)), arr) AS mapped\n\n┌─s─────┬─arr─────┬─a─┬─num─┬─mapped─┐\n│ Hello │ [1,2]   │ 1 │   1 │      2 │\n│ Hello │ [1,2]   │ 2 │   2 │      3 │\n│ World │ [3,4,5] │ 3 │   1 │      4 │\n│ World │ [3,4,5] │ 4 │   2 │      5 │\n│ World │ [3,4,5] │ 5 │   3 │      6 │\n└───────┴─────────┴───┴─────┴────────┘\n\n5 rows in set. Elapsed: 0.002 sec.\n\n:) SELECT s, arr, a, num, arrayEnumerate(arr) FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num\n\nSELECT s, arr, a, num, arrayEnumerate(arr)\nFROM arrays_test\nARRAY JOIN arr AS a, arrayEnumerate(arr) AS num\n\n┌─s─────┬─arr─────┬─a─┬─num─┬─arrayEnumerate(arr)─┐\n│ Hello │ [1,2]   │ 1 │   1 │ [1,2]               │\n│ Hello │ [1,2]   │ 2 │   2 │ [1,2]               │\n│ World │ [3,4,5] │ 3 │   1 │ [1,2,3]             │\n│ World │ [3,4,5] │ 4 │   2 │ [1,2,3]             │\n│ World │ [3,4,5] │ 5 │   3 │ [1,2,3]             │\n└───────┴─────────┴───┴─────┴─────────────────────┘\n\n5 rows in set. Elapsed: 0.002 sec.\nARRAY JOIN also works with nested data structures. Example:\n:) CREATE TABLE nested_test (s String, nest Nested(x UInt8, y UInt32)) ENGINE = Memory\n\nCREATE TABLE nested_test\n(\n    s String,\n    nest Nested(\n    x UInt8,\n    y UInt32)\n) ENGINE = Memory\n\nOk.\n\n0 rows in set. Elapsed: 0.006 sec.\n\n:) INSERT INTO nested_test VALUES ('Hello', [1,2], [10,20]), ('World', [3,4,5], [30,40,50]), ('Goodbye', [], [])\n\nINSERT INTO nested_test VALUES\n\nOk.\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT * FROM nested_test\n\nSELECT *\nFROM nested_test\n\n┌─s───────┬─nest.x──┬─nest.y─────┐\n│ Hello   │ [1,2]   │ [10,20]    │\n│ World   │ [3,4,5] │ [30,40,50] │\n│ Goodbye │ []      │ []         │\n└─────────┴─────────┴────────────┘\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest\n\nSELECT s, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN nest\n\n┌─s─────┬─nest.x─┬─nest.y─┐\n│ Hello │      1 │     10 │\n│ Hello │      2 │     20 │\n│ World │      3 │     30 │\n│ World │      4 │     40 │\n│ World │      5 │     50 │\n└───────┴────────┴────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\nWhen specifying names of nested data structures in ARRAY JOIN, the meaning is the same as ARRAY JOIN with all the array elements that it consists of. Example:\n:) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x, nest.y\n\nSELECT s, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN `nest.x`, `nest.y`\n\n┌─s─────┬─nest.x─┬─nest.y─┐\n│ Hello │      1 │     10 │\n│ Hello │      2 │     20 │\n│ World │      3 │     30 │\n│ World │      4 │     40 │\n│ World │      5 │     50 │\n└───────┴────────┴────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\nThis variation also makes sense:\n:) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x\n\nSELECT s, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN `nest.x`\n\n┌─s─────┬─nest.x─┬─nest.y─────┐\n│ Hello │      1 │ [10,20]    │\n│ Hello │      2 │ [10,20]    │\n│ World │      3 │ [30,40,50] │\n│ World │      4 │ [30,40,50] │\n│ World │      5 │ [30,40,50] │\n└───────┴────────┴────────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\nAn alias may be used for a nested data structure, in order to select either the JOIN result or the source array. Example:\n:) SELECT s, n.x, n.y, nest.x, nest.y FROM nested_test ARRAY JOIN nest AS n\n\nSELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN nest AS n\n\n┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┐\n│ Hello │   1 │  10 │ [1,2]   │ [10,20]    │\n│ Hello │   2 │  20 │ [1,2]   │ [10,20]    │\n│ World │   3 │  30 │ [3,4,5] │ [30,40,50] │\n│ World │   4 │  40 │ [3,4,5] │ [30,40,50] │\n│ World │   5 │  50 │ [3,4,5] │ [30,40,50] │\n└───────┴─────┴─────┴─────────┴────────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\nExample of using the arrayEnumerate function:\n:) SELECT s, n.x, n.y, nest.x, nest.y, num FROM nested_test ARRAY JOIN nest AS n, arrayEnumerate(nest.x) AS num\n\nSELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`, num\nFROM nested_test\nARRAY JOIN nest AS n, arrayEnumerate(`nest.x`) AS num\n\n┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┬─num─┐\n│ Hello │   1 │  10 │ [1,2]   │ [10,20]    │   1 │\n│ Hello │   2 │  20 │ [1,2]   │ [10,20]    │   2 │\n│ World │   3 │  30 │ [3,4,5] │ [30,40,50] │   1 │\n│ World │   4 │  40 │ [3,4,5] │ [30,40,50] │   2 │\n│ World │   5 │  50 │ [3,4,5] │ [30,40,50] │   3 │\n└───────┴─────┴─────┴─────────┴────────────┴─────┘\n\n5 rows in set. Elapsed: 0.002 sec.\nThe query can only specify a single ARRAY JOIN clause.\nThe corresponding conversion can be performed before the WHERE\/PREWHERE clause (if its result is needed in this clause), or after completing WHERE\/PREWHERE (to reduce the volume of calculations).\nJOIN Clause\n¶\nJoins the data in the normal\nSQL JOIN\nsense.\nNote\nNot related to\nARRAY JOIN\n.\nSELECT\n<\nexpr_list\n>\nFROM\n<\nleft_subquery\n>\n[\nGLOBAL\n]\n[\nANY\n|\nALL\n]\n[\nINNER\n|\nLEFT\n|\nRIGHT\n|\nFULL\n|\nCROSS\n]\n[\nOUTER\n]\nJOIN\n<\nright_subquery\n>\n(\nON\n<\nexpr_list\n>\n)\n|\n(\nUSING\n<\ncolumn_list\n>\n)\n...\nThe table names can be specified instead of\n<left_subquery>\nand\n<right_subquery>\n. This is equivalent to the\nSELECT * FROM table\nsubquery, except in a special case when the table has the\nJoin\nengine – an array prepared for joining.\nSupported types of\nJOIN\nINNER JOIN\n(or\nJOIN\n)\nLEFT JOIN\n(or\nLEFT OUTER JOIN\n)\nRIGHT JOIN\n(or\nRIGHT OUTER JOIN\n)\nFULL JOIN\n(or\nFULL OUTER JOIN\n)\nCROSS JOIN\n(or\n,\n)\nSee standard\nSQL JOIN\ndescription.\nANY or ALL strictness\nIf\nALL\nis specified and the right table has several matching rows, the data will be multiplied by the number of these rows. This is the normal\nJOIN\nbehavior for standard SQL.\nIf\nANY\nis specified and the right table has several matching rows, only the first one found is joined. If the right table has only one matching row, the results of\nANY\nand\nALL\nare the same.\nTo set the default strictness value, use the session configuration parameter\njoin_default_strictness\n.\nGLOBAL JOIN\nWhen using a normal\nJOIN\n, the query is sent to remote servers. Subqueries are run on each of them in order to make the right table, and the join is performed with this table. In other words, the right table is formed on each server separately.\nWhen using\nGLOBAL ... JOIN\n, first the requestor server runs a subquery to calculate the right table. This temporary table is passed to each remote server, and queries are run on them using the temporary data that was transmitted.\nBe careful when using\nGLOBAL\n. For more information, see the section\nDistributed subqueries\n.\nUsage Recommendations\nAll columns that are not needed for the\nJOIN\nare deleted from the subquery.\nWhen running a\nJOIN\n, there is no optimization of the order of execution in relation to other stages of the query. The join (a search in the right table) is run before filtering in\nWHERE\nand before aggregation. In order to explicitly set the processing order, we recommend running a\nJOIN\nsubquery with a subquery.\nExample:\nSELECT\nCounterID\n,\nhits\n,\nvisits\nFROM\n(\nSELECT\nCounterID\n,\ncount\n()\nAS\nhits\nFROM\ntest\n.\nhits\nGROUP\nBY\nCounterID\n)\nANY\nLEFT\nJOIN\n(\nSELECT\nCounterID\n,\nsum\n(\nSign\n)\nAS\nvisits\nFROM\ntest\n.\nvisits\nGROUP\nBY\nCounterID\n)\nUSING\nCounterID\nORDER\nBY\nhits\nDESC\nLIMIT\n10\n┌─CounterID─┬───hits─┬─visits─┐\n│   1143050 │ 523264 │  13665 │\n│    731962 │ 475698 │ 102716 │\n│    722545 │ 337212 │ 108187 │\n│    722889 │ 252197 │  10547 │\n│   2237260 │ 196036 │   9522 │\n│  23057320 │ 147211 │   7689 │\n│    722818 │  90109 │  17847 │\n│     48221 │  85379 │   4652 │\n│  19762435 │  77807 │   7026 │\n│    722884 │  77492 │  11056 │\n└───────────┴────────┴────────┘\nSubqueries don't allow you to set names or use them for referencing a column from a specific subquery.\nThe columns specified in\nUSING\nmust have the same names in both subqueries, and the other columns must be named differently. You can use aliases to change the names of columns in subqueries (the example uses the aliases 'hits' and 'visits').\nThe\nUSING\nclause specifies one or more columns to join, which establishes the equality of these columns. The list of columns is set without brackets. More complex join conditions are not supported.\nThe right table (the subquery result) resides in RAM. If there isn't enough memory, you can't run a\nJOIN\n.\nEach time a query is run with the same\nJOIN\n, the subquery is run again because the result is not cached. To avoid this, use the special\nJoin\ntable engine, which is a prepared array for joining that is always in RAM.\nIn some cases, it is more efficient to use\nIN\ninstead of\nJOIN\n.\nAmong the various types of\nJOIN\n, the most efficient is\nANY LEFT JOIN\n, then\nANY INNER JOIN\n. The least efficient are\nALL LEFT JOIN\nand\nALL INNER JOIN\n.\nIf you need a\nJOIN\nfor joining with dimension tables (these are relatively small tables that contain dimension properties, such as names for advertising campaigns), a\nJOIN\nmight not be very convenient due to the bulky syntax and the fact that the right table is re-accessed for every query. For such cases, there is an \"external dictionaries\" feature that you should use instead of\nJOIN\n. For more information, see the section\nExternal dictionaries\n.\nProcessing of Empty or NULL Cells\n¶\nWhile joining tables, the empty cells may appear. The setting\njoin_use_nulls\ndefine how ClickHouse fills these cells.\nIf the\nJOIN\nkeys are\nNullable\nfields, the rows where at least one of the keys has the value\nNULL\nare not joined.\nWHERE Clause\n¶\nIf there is a WHERE clause, it must contain an expression with the UInt8 type. This is usually an expression with comparison and logical operators.\nThis expression will be used for filtering data before all other transformations.\nIf indexes are supported by the database table engine, the expression is evaluated on the ability to use indexes.\nPREWHERE Clause\n¶\nThis clause has the same meaning as the WHERE clause. The difference is in which data is read from the table.\nWhen using PREWHERE, first only the columns necessary for executing PREWHERE are read. Then the other columns are read that are needed for running the query, but only those blocks where the PREWHERE expression is true.\nIt makes sense to use PREWHERE if there are filtration conditions that are used by a minority of the columns in the query, but that provide strong data filtration. This reduces the volume of data to read.\nFor example, it is useful to write PREWHERE for queries that extract a large number of columns, but that only have filtration for a few columns.\nPREWHERE is only supported by tables from the\n*MergeTree\nfamily.\nA query may simultaneously specify PREWHERE and WHERE. In this case, PREWHERE precedes WHERE.\nIf the 'optimize_move_to_prewhere' setting is set to 1 and PREWHERE is omitted, the system uses heuristics to automatically move parts of expressions from WHERE to PREWHERE.\nGROUP BY Clause\n¶\nThis is one of the most important parts of a column-oriented DBMS.\nIf there is a GROUP BY clause, it must contain a list of expressions. Each expression will be referred to here as a \"key\".\nAll the expressions in the SELECT, HAVING, and ORDER BY clauses must be calculated from keys or from aggregate functions. In other words, each column selected from the table must be used either in keys or inside aggregate functions.\nIf a query contains only table columns inside aggregate functions, the GROUP BY clause can be omitted, and aggregation by an empty set of keys is assumed.\nExample:\nSELECT\ncount\n(),\nmedian\n(\nFetchTiming\n>\n60\n?\n60\n:\nFetchTiming\n),\ncount\n()\n-\nsum\n(\nRefresh\n)\nFROM\nhits\nHowever, in contrast to standard SQL, if the table doesn't have any rows (either there aren't any at all, or there aren't any after using WHERE to filter), an empty result is returned, and not the result from one of the rows containing the initial values of aggregate functions.\nAs opposed to MySQL (and conforming to standard SQL), you can't get some value of some column that is not in a key or aggregate function (except constant expressions). To work around this, you can use the 'any' aggregate function (get the first encountered value) or 'min\/max'.\nExample:\nSELECT\ndomainWithoutWWW\n(\nURL\n)\nAS\ndomain\n,\ncount\n(),\nany\n(\nTitle\n)\nAS\ntitle\n-- getting the first occurred page header for each domain.\nFROM\nhits\nGROUP\nBY\ndomain\nFor every different key value encountered, GROUP BY calculates a set of aggregate function values.\nGROUP BY is not supported for array columns.\nA constant can't be specified as arguments for aggregate functions. Example: sum(1). Instead of this, you can get rid of the constant. Example:\ncount()\n.\nNULL processing\n¶\nFor grouping, ClickHouse interprets\nNULL\nas a value, and\nNULL=NULL\n.\nHere's an example to show what this means.\nAssume you have this table:\n┌─x─┬────y─┐\n│ 1 │    2 │\n│ 2 │ ᴺᵁᴸᴸ │\n│ 3 │    2 │\n│ 3 │    3 │\n│ 3 │ ᴺᵁᴸᴸ │\n└───┴──────┘\nThe query\nSELECT sum(x), y FROM t_null_big GROUP BY y\nresults in:\n┌─sum(x)─┬────y─┐\n│      4 │    2 │\n│      3 │    3 │\n│      5 │ ᴺᵁᴸᴸ │\n└────────┴──────┘\nYou can see that\nGROUP BY\nfor\nУ = NULL\nsummed up\nx\n, as if\nNULL\nis this value.\nIf you pass several keys to\nGROUP BY\n, the result will give you all the combinations of the selection, as if\nNULL\nwere a specific value.\nWITH TOTALS Modifier\n¶\nIf the WITH TOTALS modifier is specified, another row will be calculated. This row will have key columns containing default values (zeros or empty lines), and columns of aggregate functions with the values calculated across all the rows (the \"total\" values).\nThis extra row is output in JSON*, TabSeparated*, and Pretty* formats, separately from the other rows. In the other formats, this row is not output.\nIn JSON* formats, this row is output as a separate 'totals' field. In TabSeparated* formats, the row comes after the main result, preceded by an empty row (after the other data). In Pretty* formats, the row is output as a separate table after the main result.\nWITH TOTALS\ncan be run in different ways when HAVING is present. The behavior depends on the 'totals_mode' setting.\nBy default,\ntotals_mode = 'before_having'\n. In this case, 'totals' is calculated across all rows, including the ones that don't pass through HAVING and 'max_rows_to_group_by'.\nThe other alternatives include only the rows that pass through HAVING in 'totals', and behave differently with the setting\nmax_rows_to_group_by\nand\ngroup_by_overflow_mode = 'any'\n.\nafter_having_exclusive\n– Don't include rows that didn't pass through\nmax_rows_to_group_by\n. In other words, 'totals' will have less than or the same number of rows as it would if\nmax_rows_to_group_by\nwere omitted.\nafter_having_inclusive\n– Include all the rows that didn't pass through 'max_rows_to_group_by' in 'totals'. In other words, 'totals' will have more than or the same number of rows as it would if\nmax_rows_to_group_by\nwere omitted.\nafter_having_auto\n– Count the number of rows that passed through HAVING. If it is more than a certain amount (by default, 50%), include all the rows that didn't pass through 'max_rows_to_group_by' in 'totals'. Otherwise, do not include them.\ntotals_auto_threshold\n– By default, 0.5. The coefficient for\nafter_having_auto\n.\nIf\nmax_rows_to_group_by\nand\ngroup_by_overflow_mode = 'any'\nare not used, all variations of\nafter_having\nare the same, and you can use any of them (for example,\nafter_having_auto\n).\nYou can use WITH TOTALS in subqueries, including subqueries in the JOIN clause (in this case, the respective total values are combined).\nGROUP BY in External Memory\n¶\nYou can enable dumping temporary data to the disk to restrict memory usage during GROUP BY.\nThe\nmax_bytes_before_external_group_by\nsetting determines the threshold RAM consumption for dumping GROUP BY temporary data to the file system. If set to 0 (the default), it is disabled.\nWhen using\nmax_bytes_before_external_group_by\n, we recommend that you set max_memory_usage about twice as high. This is necessary because there are two stages to aggregation: reading the date and forming intermediate data (1) and merging the intermediate data (2). Dumping data to the file system can only occur during stage 1. If the temporary data wasn't dumped, then stage 2 might require up to the same amount of memory as in stage 1.\nFor example, if\nmax_memory_usage\nwas set to 10000000000 and you want to use external aggregation, it makes sense to set\nmax_bytes_before_external_group_by\nto 10000000000, and max_memory_usage to 20000000000. When external aggregation is triggered (if there was at least one dump of temporary data), maximum consumption of RAM is only slightly more than\nmax_bytes_before_external_group_by\n.\nWith distributed query processing, external aggregation is performed on remote servers. In order for the requestor server to use only a small amount of RAM, set\ndistributed_aggregation_memory_efficient\nto 1.\nWhen merging data flushed to the disk, as well as when merging results from remote servers when the\ndistributed_aggregation_memory_efficient\nsetting is enabled, consumes up to 1\/256 * the number of threads from the total amount of RAM.\nWhen external aggregation is enabled, if there was less than\nmax_bytes_before_external_group_by\nof data (i.e. data was not flushed), the query runs just as fast as without external aggregation. If any temporary data was flushed, the run time will be several times longer (approximately three times).\nIf you have an ORDER BY with a small LIMIT after GROUP BY, then the ORDER BY CLAUSE will not use significant amounts of RAM.\nBut if the ORDER BY doesn't have LIMIT, don't forget to enable external sorting (\nmax_bytes_before_external_sort\n).\nLIMIT N BY Clause\n¶\nLIMIT N BY COLUMNS selects the top N rows for each group of COLUMNS. LIMIT N BY is not related to LIMIT; they can both be used in the same query. The key for LIMIT N BY can contain any number of columns or expressions.\nExample:\nSELECT\ndomainWithoutWWW\n(\nURL\n)\nAS\ndomain\n,\ndomainWithoutWWW\n(\nREFERRER_URL\n)\nAS\nreferrer\n,\ndevice_type\n,\ncount\n()\ncnt\nFROM\nhits\nGROUP\nBY\ndomain\n,\nreferrer\n,\ndevice_type\nORDER\nBY\ncnt\nDESC\nLIMIT\n5\nBY\ndomain\n,\ndevice_type\nLIMIT\n100\nThe query will select the top 5 referrers for each\ndomain, device_type\npair, but not more than 100 rows (\nLIMIT n BY + LIMIT\n).\nHAVING Clause\n¶\nAllows filtering the result received after GROUP BY, similar to the WHERE clause.\nWHERE and HAVING differ in that WHERE is performed before aggregation (GROUP BY), while HAVING is performed after it.\nIf aggregation is not performed, HAVING can't be used.\nORDER BY Clause\n¶\nThe ORDER BY clause contains a list of expressions, which can each be assigned DESC or ASC (the sorting direction). If the direction is not specified, ASC is assumed. ASC is sorted in ascending order, and DESC in descending order. The sorting direction applies to a single expression, not to the entire list. Example:\nORDER BY Visits DESC, SearchPhrase\nFor sorting by String values, you can specify collation (comparison). Example:\nORDER BY SearchPhrase COLLATE 'tr'\n- for sorting by keyword in ascending order, using the Turkish alphabet, case insensitive, assuming that strings are UTF-8 encoded. COLLATE can be specified or not for each expression in ORDER BY independently. If ASC or DESC is specified, COLLATE is specified after it. When using COLLATE, sorting is always case-insensitive.\nWe only recommend using COLLATE for final sorting of a small number of rows, since sorting with COLLATE is less efficient than normal sorting by bytes.\nRows that have identical values for the list of sorting expressions are output in an arbitrary order, which can also be nondeterministic (different each time).\nIf the ORDER BY clause is omitted, the order of the rows is also undefined, and may be nondeterministic as well.\nNaN\nand\nNULL\nsorting order:\nWith the modifier\nNULLS FIRST\n— First\nNULL\n, then\nNaN\n, then other values.\nWith the modifier\nNULLS LAST\n— First the values, then\nNaN\n, then\nNULL\n.\nDefault — The same as with the\nNULLS LAST\nmodifier.\nExample:\nFor the table\n┌─x─┬────y─┐\n│ 1 │ ᴺᵁᴸᴸ │\n│ 2 │    2 │\n│ 1 │  nan │\n│ 2 │    2 │\n│ 3 │    4 │\n│ 5 │    6 │\n│ 6 │  nan │\n│ 7 │ ᴺᵁᴸᴸ │\n│ 6 │    7 │\n│ 8 │    9 │\n└───┴──────┘\nRun the query\nSELECT * FROM t_null_nan ORDER BY y NULLS FIRST\nto get:\n┌─x─┬────y─┐\n│ 1 │ ᴺᵁᴸᴸ │\n│ 7 │ ᴺᵁᴸᴸ │\n│ 1 │  nan │\n│ 6 │  nan │\n│ 2 │    2 │\n│ 2 │    2 │\n│ 3 │    4 │\n│ 5 │    6 │\n│ 6 │    7 │\n│ 8 │    9 │\n└───┴──────┘\nWhen floating point numbers are sorted, NaNs are separate from the other values. Regardless of the sorting order, NaNs come at the end. In other words, for ascending sorting they are placed as if they are larger than all the other numbers, while for descending sorting they are placed as if they are smaller than the rest.\nLess RAM is used if a small enough LIMIT is specified in addition to ORDER BY. Otherwise, the amount of memory spent is proportional to the volume of data for sorting. For distributed query processing, if GROUP BY is omitted, sorting is partially done on remote servers, and the results are merged on the requestor server. This means that for distributed sorting, the volume of data to sort can be greater than the amount of memory on a single server.\nIf there is not enough RAM, it is possible to perform sorting in external memory (creating temporary files on a disk). Use the setting\nmax_bytes_before_external_sort\nfor this purpose. If it is set to 0 (the default), external sorting is disabled. If it is enabled, when the volume of data to sort reaches the specified number of bytes, the collected data is sorted and dumped into a temporary file. After all data is read, all the sorted files are merged and the results are output. Files are written to the \/var\/lib\/clickhouse\/tmp\/ directory in the config (by default, but you can use the 'tmp_path' parameter to change this setting).\nRunning a query may use more memory than 'max_bytes_before_external_sort'. For this reason, this setting must have a value significantly smaller than 'max_memory_usage'. As an example, if your server has 128 GB of RAM and you need to run a single query, set 'max_memory_usage' to 100 GB, and 'max_bytes_before_external_sort' to 80 GB.\nExternal sorting works much less effectively than sorting in RAM.\nSELECT Clause\n¶\nThe expressions specified in the SELECT clause are analyzed after the calculations for all the clauses listed above are completed.\nMore specifically, expressions are analyzed that are above the aggregate functions, if there are any aggregate functions.\nThe aggregate functions and everything below them are calculated during aggregation (GROUP BY).\nThese expressions work as if they are applied to separate rows in the result.\nDISTINCT Clause\n¶\nIf DISTINCT is specified, only a single row will remain out of all the sets of fully matching rows in the result.\nThe result will be the same as if GROUP BY were specified across all the fields specified in SELECT without aggregate functions. But there are several differences from GROUP BY:\nDISTINCT can be applied together with GROUP BY.\nWhen ORDER BY is omitted and LIMIT is defined, the query stops running immediately after the required number of different rows has been read.\nData blocks are output as they are processed, without waiting for the entire query to finish running.\nDISTINCT is not supported if SELECT has at least one array column.\nDISTINCT\nworks with\nNULL\nas if\nNULL\nwere a specific value, and\nNULL=NULL\n. In other words, in the\nDISTINCT\nresults, different combinations with\nNULL\nonly occur once.\nLIMIT Clause\n¶\nLIMIT m\nallows you to select the first\nm\nrows from the result.\nLIMIT n\n, m allows you to select the first\nm\nrows from the result after skipping the first\nn\nrows.\nn\nand\nm\nmust be non-negative integers.\nIf there isn't an ORDER BY clause that explicitly sorts results, the result may be arbitrary and nondeterministic.\nUNION ALL Clause\n¶\nYou can use UNION ALL to combine any number of queries. Example:\nSELECT\nCounterID\n,\n1\nAS\ntable\n,\ntoInt64\n(\ncount\n())\nAS\nc\nFROM\ntest\n.\nhits\nGROUP\nBY\nCounterID\nUNION\nALL\nSELECT\nCounterID\n,\n2\nAS\ntable\n,\nsum\n(\nSign\n)\nAS\nc\nFROM\ntest\n.\nvisits\nGROUP\nBY\nCounterID\nHAVING\nc\n>\n0\nOnly UNION ALL is supported. The regular UNION (UNION DISTINCT) is not supported. If you need UNION DISTINCT, you can write SELECT DISTINCT from a subquery containing UNION ALL.\nQueries that are parts of UNION ALL can be run simultaneously, and their results can be mixed together.\nThe structure of results (the number and type of columns) must match for the queries. But the column names can differ. In this case, the column names for the final result will be taken from the first query. Type casting is performed for unions. For example, if two queries being combined have the same field with non-\nNullable\nand\nNullable\ntypes from a compatible type, the resulting\nUNION ALL\nhas a\nNullable\ntype field.\nQueries that are parts of UNION ALL can't be enclosed in brackets. ORDER BY and LIMIT are applied to separate queries, not to the final result. If you need to apply a conversion to the final result, you can put all the queries with UNION ALL in a subquery in the FROM clause.\nINTO OUTFILE Clause\n¶\nAdd the\nINTO OUTFILE filename\nclause (where filename is a string literal) to redirect query output to the specified file.\nIn contrast to MySQL, the file is created on the client side. The query will fail if a file with the same filename already exists.\nThis functionality is available in the command-line client and clickhouse-local (a query sent via HTTP interface will fail).\nThe default output format is TabSeparated (the same as in the command-line client batch mode).\nFORMAT Clause\n¶\nSpecify 'FORMAT format' to get data in any specified format.\nYou can use this for convenience, or for creating dumps.\nFor more information, see the section \"Formats\".\nIf the FORMAT clause is omitted, the default format is used, which depends on both the settings and the interface used for accessing the DB. For the HTTP interface and the command-line client in batch mode, the default format is TabSeparated. For the command-line client in interactive mode, the default format is PrettyCompact (it has attractive and compact tables).\nWhen using the command-line client, data is passed to the client in an internal efficient format. The client independently interprets the FORMAT clause of the query and formats the data itself (thus relieving the network and the server from the load).\nIN Operators\n¶\nThe\nIN\n,\nNOT IN\n,\nGLOBAL IN\n, and\nGLOBAL NOT IN\noperators are covered separately, since their functionality is quite rich.\nThe left side of the operator is either a single column or a tuple.\nExamples:\nSELECT\nUserID\nIN\n(\n123\n,\n456\n)\nFROM\n...\nSELECT\n(\nCounterID\n,\nUserID\n)\nIN\n((\n34\n,\n123\n),\n(\n101500\n,\n456\n))\nFROM\n...\nIf the left side is a single column that is in the index, and the right side is a set of constants, the system uses the index for processing the query.\nDon't list too many values explicitly (i.e. millions). If a data set is large, put it in a temporary table (for example, see the section \"External data for query processing\"), then use a subquery.\nThe right side of the operator can be a set of constant expressions, a set of tuples with constant expressions (shown in the examples above), or the name of a database table or SELECT subquery in brackets.\nIf the right side of the operator is the name of a table (for example,\nUserID IN users\n), this is equivalent to the subquery\nUserID IN (SELECT * FROM users)\n. Use this when working with external data that is sent along with the query. For example, the query can be sent together with a set of user IDs loaded to the 'users' temporary table, which should be filtered.\nIf the right side of the operator is a table name that has the Set engine (a prepared data set that is always in RAM), the data set will not be created over again for each query.\nThe subquery may specify more than one column for filtering tuples.\nExample:\nSELECT\n(\nCounterID\n,\nUserID\n)\nIN\n(\nSELECT\nCounterID\n,\nUserID\nFROM\n...)\nFROM\n...\nThe columns to the left and right of the IN operator should have the same type.\nThe IN operator and subquery may occur in any part of the query, including in aggregate functions and lambda functions.\nExample:\nSELECT\nEventDate\n,\navg\n(\nUserID\nIN\n(\nSELECT\nUserID\nFROM\ntest\n.\nhits\nWHERE\nEventDate\n=\ntoDate\n(\n'2014-03-17'\n)\n))\nAS\nratio\nFROM\ntest\n.\nhits\nGROUP\nBY\nEventDate\nORDER\nBY\nEventDate\nASC\n┌──EventDate─┬────ratio─┐\n│ 2014-03-17 │        1 │\n│ 2014-03-18 │ 0.807696 │\n│ 2014-03-19 │ 0.755406 │\n│ 2014-03-20 │ 0.723218 │\n│ 2014-03-21 │ 0.697021 │\n│ 2014-03-22 │ 0.647851 │\n│ 2014-03-23 │ 0.648416 │\n└────────────┴──────────┘\nFor each day after March 17th, count the percentage of pageviews made by users who visited the site on March 17th.\nA subquery in the IN clause is always run just one time on a single server. There are no dependent subqueries.\nNULL processing\n¶\nDuring request processing, the IN operator assumes that the result of an operation with\nNULL\nis always equal to\n0\n, regardless of whether\nNULL\nis on the right or left side of the operator.\nNULL\nvalues are not included in any dataset, do not correspond to each other and cannot be compared.\nHere is an example with the\nt_null\ntable:\n┌─x─┬────y─┐\n│ 1 │ ᴺᵁᴸᴸ │\n│ 2 │    3 │\n└───┴──────┘\nRunning the query\nSELECT x FROM t_null WHERE y IN (NULL,3)\ngives you the following result:\n┌─x─┐\n│ 2 │\n└───┘\nYou can see that the row in which\ny = NULL\nis thrown out of the query results. This is because ClickHouse can't decide whether\nNULL\nis included in the\n(NULL,3)\nset, returns\n0\nas the result of the operation, and\nSELECT\nexcludes this row from the final output.\nSELECT y IN (NULL, 3)\nFROM t_null\n\n┌─in(y, tuple(NULL, 3))─┐\n│                     0 │\n│                     1 │\n└───────────────────────┘\nDistributed Subqueries\n¶\nThere are two options for IN-s with subqueries (similar to JOINs): normal\nIN\n\/\nJOIN\nand\nGLOBAL IN\n\/\nGLOBAL JOIN\n. They differ in how they are run for distributed query processing.\nAttention\nRemember that the algorithms described below may work differently depending on the\nsettings\ndistributed_product_mode\nsetting.\nWhen using the regular IN, the query is sent to remote servers, and each of them runs the subqueries in the\nIN\nor\nJOIN\nclause.\nWhen using\nGLOBAL IN\n\/\nGLOBAL JOINs\n, first all the subqueries are run for\nGLOBAL IN\n\/\nGLOBAL JOINs\n, and the results are collected in temporary tables. Then the temporary tables are sent to each remote server, where the queries are run using this temporary data.\nFor a non-distributed query, use the regular\nIN\n\/\nJOIN\n.\nBe careful when using subqueries in the\nIN\n\/\nJOIN\nclauses for distributed query processing.\nLet's look at some examples. Assume that each server in the cluster has a normal\nlocal_table\n. Each server also has a\ndistributed_table\ntable with the\nDistributed\ntype, which looks at all the servers in the cluster.\nFor a query to the\ndistributed_table\n, the query will be sent to all the remote servers and run on them using the\nlocal_table\n.\nFor example, the query\nSELECT\nuniq\n(\nUserID\n)\nFROM\ndistributed_table\nwill be sent to all remote servers as\nSELECT\nuniq\n(\nUserID\n)\nFROM\nlocal_table\nand run on each of them in parallel, until it reaches the stage where intermediate results can be combined. Then the intermediate results will be returned to the requestor server and merged on it, and the final result will be sent to the client.\nNow let's examine a query with IN:\nSELECT\nuniq\n(\nUserID\n)\nFROM\ndistributed_table\nWHERE\nCounterID\n=\n101500\nAND\nUserID\nIN\n(\nSELECT\nUserID\nFROM\nlocal_table\nWHERE\nCounterID\n=\n34\n)\nCalculation of the intersection of audiences of two sites.\nThis query will be sent to all remote servers as\nSELECT\nuniq\n(\nUserID\n)\nFROM\nlocal_table\nWHERE\nCounterID\n=\n101500\nAND\nUserID\nIN\n(\nSELECT\nUserID\nFROM\nlocal_table\nWHERE\nCounterID\n=\n34\n)\nIn other words, the data set in the IN clause will be collected on each server independently, only across the data that is stored locally on each of the servers.\nThis will work correctly and optimally if you are prepared for this case and have spread data across the cluster servers such that the data for a single UserID resides entirely on a single server. In this case, all the necessary data will be available locally on each server. Otherwise, the result will be inaccurate. We refer to this variation of the query as \"local IN\".\nTo correct how the query works when data is spread randomly across the cluster servers, you could specify\ndistributed_table\ninside a subquery. The query would look like this:\nSELECT\nuniq\n(\nUserID\n)\nFROM\ndistributed_table\nWHERE\nCounterID\n=\n101500\nAND\nUserID\nIN\n(\nSELECT\nUserID\nFROM\ndistributed_table\nWHERE\nCounterID\n=\n34\n)\nThis query will be sent to all remote servers as\nSELECT\nuniq\n(\nUserID\n)\nFROM\nlocal_table\nWHERE\nCounterID\n=\n101500\nAND\nUserID\nIN\n(\nSELECT\nUserID\nFROM\ndistributed_table\nWHERE\nCounterID\n=\n34\n)\nThe subquery will begin running on each remote server. Since the subquery uses a distributed table, the subquery that is on each remote server will be resent to every remote server as\nSELECT\nUserID\nFROM\nlocal_table\nWHERE\nCounterID\n=\n34\nFor example, if you have a cluster of 100 servers, executing the entire query will require 10,000 elementary requests, which is generally considered unacceptable.\nIn such cases, you should always use GLOBAL IN instead of IN. Let's look at how it works for the query\nSELECT\nuniq\n(\nUserID\n)\nFROM\ndistributed_table\nWHERE\nCounterID\n=\n101500\nAND\nUserID\nGLOBAL\nIN\n(\nSELECT\nUserID\nFROM\ndistributed_table\nWHERE\nCounterID\n=\n34\n)\nThe requestor server will run the subquery\nSELECT\nUserID\nFROM\ndistributed_table\nWHERE\nCounterID\n=\n34\nand the result will be put in a temporary table in RAM. Then the request will be sent to each remote server as\nSELECT\nuniq\n(\nUserID\n)\nFROM\nlocal_table\nWHERE\nCounterID\n=\n101500\nAND\nUserID\nGLOBAL\nIN\n_data1\nand the temporary table\n_data1\nwill be sent to every remote server with the query (the name of the temporary table is implementation-defined).\nThis is more optimal than using the normal IN. However, keep the following points in mind:\nWhen creating a temporary table, data is not made unique. To reduce the volume of data transmitted over the network, specify DISTINCT in the subquery. (You don't need to do this for a normal IN.)\nThe temporary table will be sent to all the remote servers. Transmission does not account for network topology. For example, if 10 remote servers reside in a datacenter that is very remote in relation to the requestor server, the data will be sent 10 times over the channel to the remote datacenter. Try to avoid large data sets when using GLOBAL IN.\nWhen transmitting data to remote servers, restrictions on network bandwidth are not configurable. You might overload the network.\nTry to distribute data across servers so that you don't need to use GLOBAL IN on a regular basis.\nIf you need to use GLOBAL IN often, plan the location of the ClickHouse cluster so that a single group of replicas resides in no more than one data center with a fast network between them, so that a query can be processed entirely within a single data center.\nIt also makes sense to specify a local table in the\nGLOBAL IN\nclause, in case this local table is only available on the requestor server and you want to use data from it on remote servers.\nExtreme Values\n¶\nIn addition to results, you can also get minimum and maximum values for the results columns. To do this, set the\nextremes\nsetting to 1. Minimums and maximums are calculated for numeric types, dates, and dates with times. For other columns, the default values are output.\nAn extra two rows are calculated – the minimums and maximums, respectively. These extra two rows are output in JSON*, TabSeparated*, and Pretty* formats, separate from the other rows. They are not output for other formats.\nIn JSON* formats, the extreme values are output in a separate 'extremes' field. In TabSeparated* formats, the row comes after the main result, and after 'totals' if present. It is preceded by an empty row (after the other data). In Pretty* formats, the row is output as a separate table after the main result, and after 'totals' if present.\nExtreme values are calculated for rows that have passed through LIMIT. However, when using 'LIMIT offset, size', the rows before 'offset' are included in 'extremes'. In stream requests, the result may also include a small number of rows that passed through LIMIT.\nNotes\n¶\nThe\nGROUP BY\nand\nORDER BY\nclauses do not support positional arguments. This contradicts MySQL, but conforms to standard SQL.\nFor example,\nGROUP BY 1, 2\nwill be interpreted as grouping by constants (i.e. aggregation of all rows into one).\nYou can use synonyms (\nAS\naliases) in any part of a query.\nYou can put an asterisk in any part of a query instead of an expression. When the query is analyzed, the asterisk is expanded to a list of all table columns (excluding the\nMATERIALIZED\nand\nALIAS\ncolumns). There are only a few cases when using an asterisk is justified:\nWhen creating a table dump.\nFor tables containing just a few columns, such as system tables.\nFor getting information about what columns are in a table. In this case, set\nLIMIT 1\n. But it is better to use the\nDESC TABLE\nquery.\nWhen there is strong filtration on a small number of columns using\nPREWHERE\n.\nIn subqueries (since columns that aren't needed for the external query are excluded from subqueries).\nIn all other cases, we don't recommend using the asterisk, since it only gives you the drawbacks of a columnar DBMS instead of the advantages. In other words using the asterisk is not recommended.\nOriginal article","attrs_markdown":"[![](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/images\/logo.svg)](http:\/\/devdoc.net\/ \"ClickHouse Documentation\")\n\nClickHouse Documentation   SELECT\n\nType to start searching\n\nClickHouse Documentation\n\n- Introduction\n  Introduction\n  \n  - [Overview](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/ \"Overview\")\n  - [Distinctive Features of ClickHouse](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/introduction\/distinctive_features\/ \"Distinctive Features of ClickHouse\")\n  - [ClickHouse Features that Can Be Considered Disadvantages](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/introduction\/features_considered_disadvantages\/ \"ClickHouse Features that Can Be Considered Disadvantages\")\n  - [Performance](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/introduction\/performance\/ \"Performance\")\n  - [The Yandex.Metrica Task](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/introduction\/ya_metrika_task\/ \"The Yandex.Metrica Task\")\n- Getting Started\n  Getting Started\n  \n  - [Deploying and Running](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/getting_started\/ \"Deploying and Running\")\n  - Example Datasets\n    Example Datasets\n    \n    - [OnTime](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/getting_started\/example_datasets\/ontime\/ \"OnTime\")\n    - [New York Taxi Data](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/getting_started\/example_datasets\/nyc_taxi\/ \"New York Taxi Data\")\n    - [AMPLab Big Data Benchmark](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/getting_started\/example_datasets\/amplab_benchmark\/ \"AMPLab Big Data Benchmark\")\n    - [WikiStat](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/getting_started\/example_datasets\/wikistat\/ \"WikiStat\")\n    - [Terabyte Click Logs from Criteo](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/getting_started\/example_datasets\/criteo\/ \"Terabyte Click Logs from Criteo\")\n    - [Star Schema Benchmark](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/getting_started\/example_datasets\/star_schema\/ \"Star Schema Benchmark\")\n    - [Yandex.Metrica Data](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/getting_started\/example_datasets\/metrica\/ \"Yandex.Metrica Data\")\n- Interfaces\n  Interfaces\n  \n  - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/ \"Introduction\")\n  - [Command-Line Client](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/cli\/ \"Command-Line Client\")\n  - [Native Interface (TCP)](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/tcp\/ \"Native Interface (TCP)\")\n  - [HTTP Interface](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/http\/ \"HTTP Interface\")\n  - [Input and Output Formats](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/formats\/ \"Input and Output Formats\")\n  - [JDBC Driver](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/jdbc\/ \"JDBC Driver\")\n  - [ODBC Driver](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/odbc\/ \"ODBC Driver\")\n  - Third-Party\n    Third-Party\n    \n    - [Client Libraries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/third-party\/client_libraries\/ \"Client Libraries\")\n    - [Integrations](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/third-party\/integrations\/ \"Integrations\")\n    - [Visual Interfaces](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/third-party\/gui\/ \"Visual Interfaces\")\n    - [Proxies](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/interfaces\/third-party\/proxy\/ \"Proxies\")\n- Data Types\n  Data Types\n  \n  - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/ \"Introduction\")\n  - [UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/int_uint\/ \"UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64\")\n  - [Float32, Float64](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/float\/ \"Float32, Float64\")\n  - [Decimal](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/decimal\/ \"Decimal\")\n  - [Boolean](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/boolean\/ \"Boolean\")\n  - [String](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/string\/ \"String\")\n  - [FixedString(N)](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/fixedstring\/ \"FixedString(N)\")\n  - [UUID](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/uuid\/ \"UUID\")\n  - [Date](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/date\/ \"Date\")\n  - [DateTime](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/datetime\/ \"DateTime\")\n  - [Enum](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/enum\/ \"Enum\")\n  - [Array(T)](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/array\/ \"Array(T)\")\n  - [AggregateFunction(name, types\\_of\\_arguments...)](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/nested_data_structures\/aggregatefunction\/ \"AggregateFunction(name, types_of_arguments...)\")\n  - [Tuple(T1, T2, ...)](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/tuple\/ \"Tuple(T1, T2, ...)\")\n  - [Nullable](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/nullable\/ \"Nullable\")\n  - Nested Data Structures\n    Nested Data Structures\n    \n    - [Nested(Name1 Type1, Name2 Type2, ...)](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/nested_data_structures\/nested\/ \"Nested(Name1 Type1, Name2 Type2, ...)\")\n  - Special Data Types\n    Special Data Types\n    \n    - [Expression](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/special_data_types\/expression\/ \"Expression\")\n    - [Set](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/special_data_types\/set\/ \"Set\")\n    - [Nothing](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/special_data_types\/nothing\/ \"Nothing\")\n- Table Engines\n  Table Engines\n  \n  - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/ \"Introduction\")\n  - MergeTree Family\n    MergeTree Family\n    \n    - [MergeTree](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/mergetree\/ \"MergeTree\")\n    - [Data Replication](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/replication\/ \"Data Replication\")\n    - [Custom Partitioning Key](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/custom_partitioning_key\/ \"Custom Partitioning Key\")\n    - [ReplacingMergeTree](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/replacingmergetree\/ \"ReplacingMergeTree\")\n    - [SummingMergeTree](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/summingmergetree\/ \"SummingMergeTree\")\n    - [AggregatingMergeTree](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/aggregatingmergetree\/ \"AggregatingMergeTree\")\n    - [CollapsingMergeTree](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/collapsingmergetree\/ \"CollapsingMergeTree\")\n    - [VersionedCollapsingMergeTree](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/versionedcollapsingmergetree\/ \"VersionedCollapsingMergeTree\")\n    - [GraphiteMergeTree](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/graphitemergetree\/ \"GraphiteMergeTree\")\n  - Log Family\n    Log Family\n    \n    - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/log_family\/ \"Introduction\")\n    - [StripeLog](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/stripelog\/ \"StripeLog\")\n    - [Log](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/log\/ \"Log\")\n    - [TinyLog](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/tinylog\/ \"TinyLog\")\n  - Integrations\n    Integrations\n    \n    - [Kafka](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/kafka\/ \"Kafka\")\n    - [MySQL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/mysql\/ \"MySQL\")\n  - Special\n    Special\n    \n    - [Distributed](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/distributed\/ \"Distributed\")\n    - [External data](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/external_data\/ \"External data\")\n    - [Dictionary](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/dictionary\/ \"Dictionary\")\n    - [Merge](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/merge\/ \"Merge\")\n    - [File](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/file\/ \"File\")\n    - [Null](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/null\/ \"Null\")\n    - [Set](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/set\/ \"Set\")\n    - [Join](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/join\/ \"Join\")\n    - [URL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/url\/ \"URL\")\n    - [View](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/view\/ \"View\")\n    - [MaterializedView](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/materializedview\/ \"MaterializedView\")\n    - [Memory](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/memory\/ \"Memory\")\n    - [Buffer](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/buffer\/ \"Buffer\")\n- SQL Reference\n  SQL Reference\n  \n  - SELECT\n    [**SELECT**](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/ \"SELECT\")\n    Table of contents\n    \n    - [FROM Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#from-clause \"FROM Clause\")\n    - [SAMPLE Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-clause \"SAMPLE Clause\")\n      - [SAMPLE k](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-k \"SAMPLE k\")\n      - [SAMPLE n](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-n \"SAMPLE n\")\n      - [SAMPLE k OFFSET m](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-offset \"SAMPLE k OFFSET m\")\n    - [ARRAY JOIN Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-array-join-clause \"ARRAY JOIN Clause\")\n    - [JOIN Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-join \"JOIN Clause\")\n      - [Processing of Empty or NULL Cells](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#processing-of-empty-or-null-cells \"Processing of Empty or NULL Cells\")\n    - [WHERE Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#where-clause \"WHERE Clause\")\n    - [PREWHERE Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#prewhere-clause \"PREWHERE Clause\")\n    - [GROUP BY Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-group-by-clause \"GROUP BY Clause\")\n      - [NULL processing](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#null-processing \"NULL processing\")\n      - [WITH TOTALS Modifier](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#with-totals-modifier \"WITH TOTALS Modifier\")\n      - [GROUP BY in External Memory](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-group-by-in-external-memory \"GROUP BY in External Memory\")\n    - [LIMIT N BY Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#limit-n-by-clause \"LIMIT N BY Clause\")\n    - [HAVING Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#having-clause \"HAVING Clause\")\n    - [ORDER BY Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#order-by-clause \"ORDER BY Clause\")\n    - [SELECT Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-clause \"SELECT Clause\")\n    - [DISTINCT Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distinct \"DISTINCT Clause\")\n    - [LIMIT Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#limit-clause \"LIMIT Clause\")\n    - [UNION ALL Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#union-all-clause \"UNION ALL Clause\")\n    - [INTO OUTFILE Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#into-outfile-clause \"INTO OUTFILE Clause\")\n    - [FORMAT Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#format-clause \"FORMAT Clause\")\n    - [IN Operators](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-in-operators \"IN Operators\")\n      - [NULL processing](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#null-processing_1 \"NULL processing\")\n      - [Distributed Subqueries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distributed-subqueries \"Distributed Subqueries\")\n    - [Extreme Values](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#extreme-values \"Extreme Values\")\n    - [Notes](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#notes \"Notes\")\n  - [INSERT INTO](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/insert_into\/ \"INSERT INTO\")\n  - [CREATE](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/create\/ \"CREATE\")\n  - [ALTER](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/alter\/ \"ALTER\")\n  - [Other Kinds of Queries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/misc\/ \"Other Kinds of Queries\")\n  - Functions\n    Functions\n    \n    - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/ \"Introduction\")\n    - [Arithmetic](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/arithmetic_functions\/ \"Arithmetic\")\n    - [Comparison](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/comparison_functions\/ \"Comparison\")\n    - [Logical](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/logical_functions\/ \"Logical\")\n    - [Type Conversion](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/type_conversion_functions\/ \"Type Conversion\")\n    - [Working with Dates and Times](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/date_time_functions\/ \"Working with Dates and Times\")\n    - [Working with strings](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/string_functions\/ \"Working with strings\")\n    - [For Searching Strings](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/string_search_functions\/ \"For Searching Strings\")\n    - [For Replacing in Strings](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/string_replace_functions\/ \"For Replacing in Strings\")\n    - [Conditional](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/conditional_functions\/ \"Conditional \")\n    - [Mathematical](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/math_functions\/ \"Mathematical\")\n    - [Rounding](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/rounding_functions\/ \"Rounding\")\n    - [Working with Arrays](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/array_functions\/ \"Working with Arrays\")\n    - [Splitting and Merging Strings and Arrays](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/splitting_merging_functions\/ \"Splitting and Merging Strings and Arrays\")\n    - [Bit](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/bit_functions\/ \"Bit\")\n    - [Bitmap functions](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/bitmap_functions\/ \"Bitmap functions\")\n    - [Hash](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/hash_functions\/ \"Hash\")\n    - [Generating Pseudo-Random Numbers](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/random_functions\/ \"Generating Pseudo-Random Numbers\")\n    - [Encoding](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/encoding_functions\/ \"Encoding\")\n    - [Working with UUID](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/uuid_functions\/ \"Working with UUID\")\n    - [Working with URLs](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/url_functions\/ \"Working with URLs\")\n    - [Working with IP Addresses](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/ip_address_functions\/ \"Working with IP Addresses\")\n    - [Working with JSON.](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/json_functions\/ \"Working with JSON.\")\n    - [Higher-Order](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/higher_order_functions\/ \"Higher-Order\")\n    - [Working with External Dictionaries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/ext_dict_functions\/ \"Working with External Dictionaries\")\n    - [Working with Yandex.Metrica Dictionaries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/ym_dict_functions\/ \"Working with Yandex.Metrica Dictionaries\")\n    - [Implementing the IN Operator](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/in_functions\/ \"Implementing the IN Operator\")\n    - [arrayJoin](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/array_join\/ \"arrayJoin\")\n    - [Working with geographical coordinates](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/geo\/ \"Working with geographical coordinates\")\n    - [Working with Nullable arguments](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/functions_for_nulls\/ \"Working with Nullable arguments\")\n    - [Other](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/functions\/other_functions\/ \"Other\")\n  - Aggregate Functions\n    Aggregate Functions\n    \n    - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/agg_functions\/ \"Introduction\")\n    - [Reference](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/agg_functions\/reference\/ \"Reference\")\n    - [Aggregate function combinators](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/agg_functions\/combinators\/ \"Aggregate function combinators\")\n    - [Parametric aggregate functions](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/agg_functions\/parametric_functions\/ \"Parametric aggregate functions\")\n  - Table Functions\n    Table Functions\n    \n    - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/table_functions\/ \"Introduction\")\n    - [file](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/table_functions\/file\/ \"file\")\n    - [merge](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/table_functions\/merge\/ \"merge\")\n    - [numbers](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/table_functions\/numbers\/ \"numbers\")\n    - [remote](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/table_functions\/remote\/ \"remote\")\n    - [url](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/table_functions\/url\/ \"url\")\n    - [jdbc](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/table_functions\/jdbc\/ \"jdbc\")\n  - Dictionaries\n    Dictionaries\n    \n    - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/ \"Introduction\")\n    - External Dictionaries\n      External Dictionaries\n      \n      - [General Description](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/external_dicts\/ \"General Description\")\n      - [Configuring an External Dictionary](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/external_dicts_dict\/ \"Configuring an External Dictionary\")\n      - [Storing Dictionaries in Memory](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/external_dicts_dict_layout\/ \"Storing Dictionaries in Memory\")\n      - [Dictionary Updates](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/external_dicts_dict_lifetime\/ \"Dictionary Updates\")\n      - [Sources of External Dictionaries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/external_dicts_dict_sources\/ \"Sources of External Dictionaries\")\n      - [Dictionary Key and Fields](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/external_dicts_dict_structure\/ \"Dictionary Key and Fields\")\n    - [Internal Dictionaries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/internal_dicts\/ \"Internal Dictionaries\")\n  - [Operators](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/operators\/ \"Operators\")\n  - [General Syntax](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/syntax\/ \"General Syntax\")\n- Operations\n  Operations\n  \n  - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/ \"Introduction\")\n  - [Requirements](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/requirements\/ \"Requirements\")\n  - [Monitoring](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/monitoring\/ \"Monitoring\")\n  - [Troubleshooting](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/troubleshooting\/ \"Troubleshooting\")\n  - [Usage Recommendations](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/tips\/ \"Usage Recommendations\")\n  - [ClickHouse Update](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/update\/ \"ClickHouse Update\")\n  - [Access Rights](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/access_rights\/ \"Access Rights\")\n  - [Data Backup](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/backup\/ \"Data Backup\")\n  - [Configuration Files](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/configuration_files\/ \"Configuration Files\")\n  - [Quotas](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/quotas\/ \"Quotas\")\n  - [System Tables](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/system_tables\/ \"System Tables\")\n  - Server Configuration Parameters\n    Server Configuration Parameters\n    \n    - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/server_settings\/ \"Introduction\")\n    - [Server Settings](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/server_settings\/settings\/ \"Server Settings\")\n  - Settings\n    Settings\n    \n    - [Introduction](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/ \"Introduction\")\n    - [Permissions for Queries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/permissions_for_queries\/ \"Permissions for Queries\")\n    - [Restrictions on Query Complexity](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/query_complexity\/ \"Restrictions on Query Complexity\")\n    - [Settings](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/settings\/ \"Settings\")\n    - [Settings Profiles](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/settings_profiles\/ \"Settings Profiles\")\n  - Utilities\n    Utilities\n    \n    - [Overview](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/utils\/ \"Overview\")\n    - [clickhouse-copier](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/utils\/clickhouse-copier\/ \"clickhouse-copier\")\n    - [clickhouse-local](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/utils\/clickhouse-local\/ \"clickhouse-local\")\n- F.A.Q.\n  F.A.Q.\n  \n  - [General Questions](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/faq\/general\/ \"General Questions\")\n- Development\n  Development\n  \n  - [Overview of ClickHouse Architecture](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/development\/architecture\/ \"Overview of ClickHouse Architecture\")\n  - [How to Build ClickHouse on Linux](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/development\/build\/ \"How to Build ClickHouse on Linux\")\n  - [How to Build ClickHouse on Mac OS X](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/development\/build_osx\/ \"How to Build ClickHouse on Mac OS X\")\n  - [How to Write C++ code](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/development\/style\/ \"How to Write C++ code\")\n  - [How to Run ClickHouse Tests](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/development\/tests\/ \"How to Run ClickHouse Tests\")\n  - [Third-Party Libraries Used](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/development\/contrib\/ \"Third-Party Libraries Used\")\n- What's New\n  What's New\n  \n  - [Roadmap](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/roadmap\/ \"Roadmap\")\n  - [Changelog](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/changelog\/ \"Changelog\")\n  - [Security Changelog](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/security_changelog\/ \"Security Changelog\")\n\n- [Single page version](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/single\/)\n- [PDF version](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/single\/clickhouse_en.pdf)\n- [ClickHouse source code](https:\/\/github.com\/yandex\/ClickHouse\/)\n\nTable of contents\n\n- [FROM Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#from-clause \"FROM Clause\")\n- [SAMPLE Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-clause \"SAMPLE Clause\")\n  - [SAMPLE k](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-k \"SAMPLE k\")\n  - [SAMPLE n](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-n \"SAMPLE n\")\n  - [SAMPLE k OFFSET m](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-offset \"SAMPLE k OFFSET m\")\n- [ARRAY JOIN Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-array-join-clause \"ARRAY JOIN Clause\")\n- [JOIN Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-join \"JOIN Clause\")\n  - [Processing of Empty or NULL Cells](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#processing-of-empty-or-null-cells \"Processing of Empty or NULL Cells\")\n- [WHERE Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#where-clause \"WHERE Clause\")\n- [PREWHERE Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#prewhere-clause \"PREWHERE Clause\")\n- [GROUP BY Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-group-by-clause \"GROUP BY Clause\")\n  - [NULL processing](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#null-processing \"NULL processing\")\n  - [WITH TOTALS Modifier](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#with-totals-modifier \"WITH TOTALS Modifier\")\n  - [GROUP BY in External Memory](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-group-by-in-external-memory \"GROUP BY in External Memory\")\n- [LIMIT N BY Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#limit-n-by-clause \"LIMIT N BY Clause\")\n- [HAVING Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#having-clause \"HAVING Clause\")\n- [ORDER BY Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#order-by-clause \"ORDER BY Clause\")\n- [SELECT Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-clause \"SELECT Clause\")\n- [DISTINCT Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distinct \"DISTINCT Clause\")\n- [LIMIT Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#limit-clause \"LIMIT Clause\")\n- [UNION ALL Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#union-all-clause \"UNION ALL Clause\")\n- [INTO OUTFILE Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#into-outfile-clause \"INTO OUTFILE Clause\")\n- [FORMAT Clause](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#format-clause \"FORMAT Clause\")\n- [IN Operators](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-in-operators \"IN Operators\")\n  - [NULL processing](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#null-processing_1 \"NULL processing\")\n  - [Distributed Subqueries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distributed-subqueries \"Distributed Subqueries\")\n- [Extreme Values](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#extreme-values \"Extreme Values\")\n- [Notes](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#notes \"Notes\")\n\n[](https:\/\/github.com\/yandex\/ClickHouse\/edit\/master\/docs\/en\/query_language\/select.md \"Edit this page\")\n\n# SELECT Queries Syntax[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-queries-syntax \"Permanent link\")\n`SELECT` performs data retrieval.\n```\nSELECT [DISTINCT] expr_list\n    [FROM [db.]table | (subquery) | table_function] [FINAL]\n    [SAMPLE sample_coeff]\n    [ARRAY JOIN ...]\n    [GLOBAL] [ANY|ALL] [INNER|LEFT|RIGHT|FULL|CROSS] [OUTER] JOIN (subquery)|table USING columns_list\n    [PREWHERE expr]\n    [WHERE expr]\n    [GROUP BY expr_list] [WITH TOTALS]\n    [HAVING expr]\n    [ORDER BY expr_list]\n    [LIMIT [n, ]m]\n    [UNION ALL ...]\n    [INTO OUTFILE filename]\n    [FORMAT format]\n    [LIMIT n BY columns]\n```\nAll the clauses are optional, except for the required list of expressions immediately after SELECT. The clauses below are described in almost the same order as in the query execution conveyor.\n\nIf the query omits the `DISTINCT`, `GROUP BY` and `ORDER BY` clauses and the `IN` and `JOIN` subqueries, the query will be completely stream processed, using O(1) amount of RAM. Otherwise, the query might consume a lot of RAM if the appropriate restrictions are not specified: `max_memory_usage`, `max_rows_to_group_by`, `max_rows_to_sort`, `max_rows_in_distinct`, `max_bytes_in_distinct`, `max_rows_in_set`, `max_bytes_in_set`, `max_rows_in_join`, `max_bytes_in_join`, `max_bytes_before_external_sort`, `max_bytes_before_external_group_by`. For more information, see the section \"Settings\". It is possible to use external sorting (saving temporary tables to a disk) and external aggregation. `The system does not have \"merge join\"`.\n\n### FROM Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#from-clause \"Permanent link\")\nIf the FROM clause is omitted, data will be read from the `system.one` table. The 'system.one' table contains exactly one row (this table fulfills the same purpose as the DUAL table found in other DBMSs).\n\nThe FROM clause specifies the table to read data from, or a subquery, or a table function; ARRAY JOIN and the regular JOIN may also be included (see below).\n\nInstead of a table, the SELECT subquery may be specified in brackets. In this case, the subquery processing pipeline will be built into the processing pipeline of an external query. In contrast to standard SQL, a synonym does not need to be specified after a subquery. For compatibility, it is possible to write 'AS name' after a subquery, but the specified name isn't used anywhere.\n\nA table function may be specified instead of a table. For more information, see the section \"Table functions\".\n\nTo execute a query, all the columns listed in the query are extracted from the appropriate table. Any columns not needed for the external query are thrown out of the subqueries. If a query does not list any columns (for example, SELECT count() FROM t), some column is extracted from the table anyway (the smallest one is preferred), in order to calculate the number of rows.\n\nThe FINAL modifier can be used only for a SELECT from a CollapsingMergeTree table. When you specify FINAL, data is selected fully \"collapsed\". Keep in mind that using FINAL leads to a selection that includes columns related to the primary key, in addition to the columns specified in the SELECT. Additionally, the query will be executed in a single stream, and data will be merged during query execution. This means that when using FINAL, the query is processed more slowly. In most cases, you should avoid using FINAL. For more information, see the section \"CollapsingMergeTree engine\".\n\n### SAMPLE Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-clause \"Permanent link\")\nThe `SAMPLE` clause allows for approximated query processing. Approximated query processing is only supported by the tables in the `MergeTree` family, and only if the sampling expression was specified during table creation (see [MergeTree engine](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/mergetree\/)).\n\nThe features of data sampling are listed below:\n\n- Data sampling is a deterministic mechanism. The result of the same `SELECT .. SAMPLE` query is always the same.\n- Sampling works consistently for different tables. For tables with a single sampling key, a sample with the same coefficient always selects the same subset of possible data. For example, a sample of user IDs takes rows with the same subset of all the possible user IDs from different tables. This allows using the sample in subqueries in the `IN` clause, as well as for manually correlating results of different queries with samples.\n- Sampling allows reading less data from a disk. Note that for this you must specify the sampling key correctly. For more details see [Creating a MergeTree Table](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/mergetree\/#table_engine-mergetree-creating-a-table).\n\nThe `SAMPLE` clause can be specified in several ways:\n\n- `SAMPLE k`, where `k` is a decimal number from 0 to 1. The query is executed on `k` percent of data. For example, `SAMPLE 0.1` runs the query on 10% of data. [Read more](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-k)\n- `SAMPLE n`, where `n` is a sufficiently large integer. The query is executed on a sample of at least `n` rows (but not significantly more than this). For example, `SAMPLE 10000000` runs the query on a minimum of 10,000,000 rows. [Read more](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-n)\n- `SAMPLE k OFFSET m` where `k` and `m` are numbers from 0 to 1. The query is executed on a sample of `k` percent of the data. The data used for the sample is offset by `m` percent. [Read more](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-offset)\n#### SAMPLE k[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-k \"Permanent link\")\nIn a `SAMPLE k` clause, `k` is a percent amount of data that the sample is taken from. The example is shown below:\n```\nSELECT\n    Title,\n    count() * 10 AS PageViews\nFROM hits_distributed\nSAMPLE 0.1\nWHERE\n    CounterID = 34\nGROUP BY Title\nORDER BY PageViews DESC LIMIT 1000\n```\nIn this example, the query is executed on a sample from 0.1 (10%) of data. Values of aggregate functions are not corrected automatically, so to get an approximate result, the value 'count()' is manually multiplied by 10.\n\n#### SAMPLE n[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-n \"Permanent link\")\nIn this case, the query is executed on a sample of at least `n` rows, where `n` is a sufficiently large integer. For example, `SAMPLE 10000000`.\n\nSince the minimum unit for data reading is one granule (its size is set by the `index_granularity` setting), it makes sense to set a sample that is much larger than the size of the granule.\n\nWhen using the `SAMPLE n` clause, the relative coefficient is calculated dynamically. Since you do not know which relative percent of data was processed, you do not know the coefficient the aggregate functions should be multiplied by (for example, you do not know if the `SAMPLE 1000000` was taken from a set of 10,000,000 rows or from a set of 1,000,000,000 rows). In this case, use the `_sample_factor` column to get the approximate result.\n\nThe `_sample_factor` is a virtual column that ClickHouse stores relative coefficients in. This column is created automatically when you create a table with the specified sampling key. The usage example is shown below:\n```\nSELECT sum(Duration * _sample_factor)\nFROM visits\nSAMPLE 10000000\n```\nIf you need to get the approximate count of rows in a `SELECT .. SAMPLE n` query, get the sum() of `_sample_factor` column instead of counting `count(column * _sample_factor)` value. For example:\n```\nSELECT sum(_sample_factor)\nFROM visits\nSAMPLE 10000000\n```\nNote that to calculate the average in a `SELECT .. SAMPLE n` query, you do not need to use `_sample_factor` column:\n```\nSELECT avg(Duration)\nFROM visits\nSAMPLE 10000000\n```\n#### SAMPLE k OFFSET m[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-offset \"Permanent link\")\nYou can specify the `SAMPLE k OFFSET m` clause, where `k` and `m` are numbers from 0 to 1. Examples are shown below.\n\nExample 1.\n```\nSAMPLE 1\/10\n```\nIn this example, the sample is the 1\/10th of all data:\n\n`[++------------------]`\n\nExample 2.\n```\nSAMPLE 1\/10 OFFSET 1\/2\n```\nHere, the sample of 10% is taken from the second half of data.\n\n`[----------++--------]`\n\n### ARRAY JOIN Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-array-join-clause \"Permanent link\")\nAllows executing JOIN with an array or nested data structure. The intent is similar to the 'arrayJoin' function, but its functionality is broader.\n\n`ARRAY JOIN` is essentially `INNER JOIN` with an array. Example:\n```\n:) CREATE TABLE arrays_test (s String, arr Array(UInt8)) ENGINE = Memory\n\nCREATE TABLE arrays_test\n(\n    s String,\n    arr Array(UInt8)\n) ENGINE = Memory\n\nOk.\n\n0 rows in set. Elapsed: 0.001 sec.\n\n:) INSERT INTO arrays_test VALUES ('Hello', [1,2]), ('World', [3,4,5]), ('Goodbye', [])\n\nINSERT INTO arrays_test VALUES\n\nOk.\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT * FROM arrays_test\n\nSELECT *\nFROM arrays_test\n\n┌─s───────┬─arr─────┐\n│ Hello   │ [1,2]   │\n│ World   │ [3,4,5] │\n│ Goodbye │ []      │\n└─────────┴─────────┘\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT s, arr FROM arrays_test ARRAY JOIN arr\n\nSELECT s, arr\nFROM arrays_test\nARRAY JOIN arr\n\n┌─s─────┬─arr─┐\n│ Hello │   1 │\n│ Hello │   2 │\n│ World │   3 │\n│ World │   4 │\n│ World │   5 │\n└───────┴─────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nAn alias can be specified for an array in the ARRAY JOIN clause. In this case, an array item can be accessed by this alias, but the array itself by the original name. Example:\n```\n:) SELECT s, arr, a FROM arrays_test ARRAY JOIN arr AS a\n\nSELECT s, arr, a\nFROM arrays_test\nARRAY JOIN arr AS a\n\n┌─s─────┬─arr─────┬─a─┐\n│ Hello │ [1,2]   │ 1 │\n│ Hello │ [1,2]   │ 2 │\n│ World │ [3,4,5] │ 3 │\n│ World │ [3,4,5] │ 4 │\n│ World │ [3,4,5] │ 5 │\n└───────┴─────────┴───┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nMultiple arrays of the same size can be comma-separated in the ARRAY JOIN clause. In this case, JOIN is performed with them simultaneously (the direct sum, not the direct product). Example:\n```\n:) SELECT s, arr, a, num, mapped FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(x -> x + 1, arr) AS mapped\n\nSELECT s, arr, a, num, mapped\nFROM arrays_test\nARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(lambda(tuple(x), plus(x, 1)), arr) AS mapped\n\n┌─s─────┬─arr─────┬─a─┬─num─┬─mapped─┐\n│ Hello │ [1,2]   │ 1 │   1 │      2 │\n│ Hello │ [1,2]   │ 2 │   2 │      3 │\n│ World │ [3,4,5] │ 3 │   1 │      4 │\n│ World │ [3,4,5] │ 4 │   2 │      5 │\n│ World │ [3,4,5] │ 5 │   3 │      6 │\n└───────┴─────────┴───┴─────┴────────┘\n\n5 rows in set. Elapsed: 0.002 sec.\n\n:) SELECT s, arr, a, num, arrayEnumerate(arr) FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num\n\nSELECT s, arr, a, num, arrayEnumerate(arr)\nFROM arrays_test\nARRAY JOIN arr AS a, arrayEnumerate(arr) AS num\n\n┌─s─────┬─arr─────┬─a─┬─num─┬─arrayEnumerate(arr)─┐\n│ Hello │ [1,2]   │ 1 │   1 │ [1,2]               │\n│ Hello │ [1,2]   │ 2 │   2 │ [1,2]               │\n│ World │ [3,4,5] │ 3 │   1 │ [1,2,3]             │\n│ World │ [3,4,5] │ 4 │   2 │ [1,2,3]             │\n│ World │ [3,4,5] │ 5 │   3 │ [1,2,3]             │\n└───────┴─────────┴───┴─────┴─────────────────────┘\n\n5 rows in set. Elapsed: 0.002 sec.\n```\nARRAY JOIN also works with nested data structures. Example:\n```\n:) CREATE TABLE nested_test (s String, nest Nested(x UInt8, y UInt32)) ENGINE = Memory\n\nCREATE TABLE nested_test\n(\n    s String,\n    nest Nested(\n    x UInt8,\n    y UInt32)\n) ENGINE = Memory\n\nOk.\n\n0 rows in set. Elapsed: 0.006 sec.\n\n:) INSERT INTO nested_test VALUES ('Hello', [1,2], [10,20]), ('World', [3,4,5], [30,40,50]), ('Goodbye', [], [])\n\nINSERT INTO nested_test VALUES\n\nOk.\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT * FROM nested_test\n\nSELECT *\nFROM nested_test\n\n┌─s───────┬─nest.x──┬─nest.y─────┐\n│ Hello   │ [1,2]   │ [10,20]    │\n│ World   │ [3,4,5] │ [30,40,50] │\n│ Goodbye │ []      │ []         │\n└─────────┴─────────┴────────────┘\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest\n\nSELECT s, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN nest\n\n┌─s─────┬─nest.x─┬─nest.y─┐\n│ Hello │      1 │     10 │\n│ Hello │      2 │     20 │\n│ World │      3 │     30 │\n│ World │      4 │     40 │\n│ World │      5 │     50 │\n└───────┴────────┴────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nWhen specifying names of nested data structures in ARRAY JOIN, the meaning is the same as ARRAY JOIN with all the array elements that it consists of. Example:\n```\n:) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x, nest.y\n\nSELECT s, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN `nest.x`, `nest.y`\n\n┌─s─────┬─nest.x─┬─nest.y─┐\n│ Hello │      1 │     10 │\n│ Hello │      2 │     20 │\n│ World │      3 │     30 │\n│ World │      4 │     40 │\n│ World │      5 │     50 │\n└───────┴────────┴────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nThis variation also makes sense:\n```\n:) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x\n\nSELECT s, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN `nest.x`\n\n┌─s─────┬─nest.x─┬─nest.y─────┐\n│ Hello │      1 │ [10,20]    │\n│ Hello │      2 │ [10,20]    │\n│ World │      3 │ [30,40,50] │\n│ World │      4 │ [30,40,50] │\n│ World │      5 │ [30,40,50] │\n└───────┴────────┴────────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nAn alias may be used for a nested data structure, in order to select either the JOIN result or the source array. Example:\n```\n:) SELECT s, n.x, n.y, nest.x, nest.y FROM nested_test ARRAY JOIN nest AS n\n\nSELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN nest AS n\n\n┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┐\n│ Hello │   1 │  10 │ [1,2]   │ [10,20]    │\n│ Hello │   2 │  20 │ [1,2]   │ [10,20]    │\n│ World │   3 │  30 │ [3,4,5] │ [30,40,50] │\n│ World │   4 │  40 │ [3,4,5] │ [30,40,50] │\n│ World │   5 │  50 │ [3,4,5] │ [30,40,50] │\n└───────┴─────┴─────┴─────────┴────────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nExample of using the arrayEnumerate function:\n```\n:) SELECT s, n.x, n.y, nest.x, nest.y, num FROM nested_test ARRAY JOIN nest AS n, arrayEnumerate(nest.x) AS num\n\nSELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`, num\nFROM nested_test\nARRAY JOIN nest AS n, arrayEnumerate(`nest.x`) AS num\n\n┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┬─num─┐\n│ Hello │   1 │  10 │ [1,2]   │ [10,20]    │   1 │\n│ Hello │   2 │  20 │ [1,2]   │ [10,20]    │   2 │\n│ World │   3 │  30 │ [3,4,5] │ [30,40,50] │   1 │\n│ World │   4 │  40 │ [3,4,5] │ [30,40,50] │   2 │\n│ World │   5 │  50 │ [3,4,5] │ [30,40,50] │   3 │\n└───────┴─────┴─────┴─────────┴────────────┴─────┘\n\n5 rows in set. Elapsed: 0.002 sec.\n```\nThe query can only specify a single ARRAY JOIN clause.\n\nThe corresponding conversion can be performed before the WHERE\/PREWHERE clause (if its result is needed in this clause), or after completing WHERE\/PREWHERE (to reduce the volume of calculations).\n\n### JOIN Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-join \"Permanent link\")\nJoins the data in the normal [SQL JOIN](https:\/\/en.wikipedia.org\/wiki\/Join_\\(SQL\\)) sense.\n\nNote\n\nNot related to [ARRAY JOIN](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-array-join-clause).\n```\nSELECT <expr_list>\nFROM <left_subquery>\n[GLOBAL] [ANY|ALL] [INNER|LEFT|RIGHT|FULL|CROSS] [OUTER] JOIN <right_subquery>\n(ON <expr_list>)|(USING <column_list>) ...\n```\nThe table names can be specified instead of `<left_subquery>` and `<right_subquery>`. This is equivalent to the `SELECT * FROM table` subquery, except in a special case when the table has the [Join](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/join\/) engine – an array prepared for joining.\n\n**Supported types of `JOIN`**\n\n- `INNER JOIN` (or `JOIN`)\n- `LEFT JOIN` (or `LEFT OUTER JOIN`)\n- `RIGHT JOIN` (or `RIGHT OUTER JOIN`)\n- `FULL JOIN` (or `FULL OUTER JOIN`)\n- `CROSS JOIN` (or `,` )\n\nSee standard [SQL JOIN](https:\/\/en.wikipedia.org\/wiki\/Join_\\(SQL\\)) description.\n\n**ANY or ALL strictness**\n\nIf `ALL` is specified and the right table has several matching rows, the data will be multiplied by the number of these rows. This is the normal `JOIN` behavior for standard SQL. If `ANY` is specified and the right table has several matching rows, only the first one found is joined. If the right table has only one matching row, the results of `ANY` and `ALL` are the same.\n\nTo set the default strictness value, use the session configuration parameter [join\\_default\\_strictness](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/settings\/#settings-join_default_strictness).\n\n**GLOBAL JOIN**\n\nWhen using a normal `JOIN`, the query is sent to remote servers. Subqueries are run on each of them in order to make the right table, and the join is performed with this table. In other words, the right table is formed on each server separately.\n\nWhen using `GLOBAL ... JOIN`, first the requestor server runs a subquery to calculate the right table. This temporary table is passed to each remote server, and queries are run on them using the temporary data that was transmitted.\n\nBe careful when using `GLOBAL`. For more information, see the section [Distributed subqueries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distributed-subqueries).\n\n**Usage Recommendations**\n\nAll columns that are not needed for the `JOIN` are deleted from the subquery.\n\nWhen running a `JOIN`, there is no optimization of the order of execution in relation to other stages of the query. The join (a search in the right table) is run before filtering in `WHERE` and before aggregation. In order to explicitly set the processing order, we recommend running a `JOIN` subquery with a subquery.\n\nExample:\n```\nSELECT\n    CounterID,\n    hits,\n    visits\nFROM\n(\n    SELECT\n        CounterID,\n        count() AS hits\n    FROM test.hits\n    GROUP BY CounterID\n) ANY LEFT JOIN\n(\n    SELECT\n        CounterID,\n        sum(Sign) AS visits\n    FROM test.visits\n    GROUP BY CounterID\n) USING CounterID\nORDER BY hits DESC\nLIMIT 10\n```\n```\n┌─CounterID─┬───hits─┬─visits─┐\n│   1143050 │ 523264 │  13665 │\n│    731962 │ 475698 │ 102716 │\n│    722545 │ 337212 │ 108187 │\n│    722889 │ 252197 │  10547 │\n│   2237260 │ 196036 │   9522 │\n│  23057320 │ 147211 │   7689 │\n│    722818 │  90109 │  17847 │\n│     48221 │  85379 │   4652 │\n│  19762435 │  77807 │   7026 │\n│    722884 │  77492 │  11056 │\n└───────────┴────────┴────────┘\n```\nSubqueries don't allow you to set names or use them for referencing a column from a specific subquery. The columns specified in `USING` must have the same names in both subqueries, and the other columns must be named differently. You can use aliases to change the names of columns in subqueries (the example uses the aliases 'hits' and 'visits').\n\nThe `USING` clause specifies one or more columns to join, which establishes the equality of these columns. The list of columns is set without brackets. More complex join conditions are not supported.\n\nThe right table (the subquery result) resides in RAM. If there isn't enough memory, you can't run a `JOIN`.\n\nEach time a query is run with the same `JOIN`, the subquery is run again because the result is not cached. To avoid this, use the special [Join](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/join\/) table engine, which is a prepared array for joining that is always in RAM.\n\nIn some cases, it is more efficient to use `IN` instead of `JOIN`. Among the various types of `JOIN`, the most efficient is `ANY LEFT JOIN`, then `ANY INNER JOIN`. The least efficient are `ALL LEFT JOIN` and `ALL INNER JOIN`.\n\nIf you need a `JOIN` for joining with dimension tables (these are relatively small tables that contain dimension properties, such as names for advertising campaigns), a `JOIN` might not be very convenient due to the bulky syntax and the fact that the right table is re-accessed for every query. For such cases, there is an \"external dictionaries\" feature that you should use instead of `JOIN`. For more information, see the section [External dictionaries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/external_dicts\/).\n\n#### Processing of Empty or NULL Cells[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#processing-of-empty-or-null-cells \"Permanent link\")\nWhile joining tables, the empty cells may appear. The setting [join\\_use\\_nulls](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/settings\/#settings-join_use_nulls) define how ClickHouse fills these cells.\n\nIf the `JOIN` keys are [Nullable](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/nullable\/) fields, the rows where at least one of the keys has the value [NULL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/syntax\/#null-literal) are not joined.\n\n### WHERE Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#where-clause \"Permanent link\")\nIf there is a WHERE clause, it must contain an expression with the UInt8 type. This is usually an expression with comparison and logical operators. This expression will be used for filtering data before all other transformations.\n\nIf indexes are supported by the database table engine, the expression is evaluated on the ability to use indexes.\n\n### PREWHERE Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#prewhere-clause \"Permanent link\")\nThis clause has the same meaning as the WHERE clause. The difference is in which data is read from the table. When using PREWHERE, first only the columns necessary for executing PREWHERE are read. Then the other columns are read that are needed for running the query, but only those blocks where the PREWHERE expression is true.\n\nIt makes sense to use PREWHERE if there are filtration conditions that are used by a minority of the columns in the query, but that provide strong data filtration. This reduces the volume of data to read.\n\nFor example, it is useful to write PREWHERE for queries that extract a large number of columns, but that only have filtration for a few columns.\n\nPREWHERE is only supported by tables from the `*MergeTree` family.\n\nA query may simultaneously specify PREWHERE and WHERE. In this case, PREWHERE precedes WHERE.\n\nIf the 'optimize\\_move\\_to\\_prewhere' setting is set to 1 and PREWHERE is omitted, the system uses heuristics to automatically move parts of expressions from WHERE to PREWHERE.\n\n### GROUP BY Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-group-by-clause \"Permanent link\")\nThis is one of the most important parts of a column-oriented DBMS.\n\nIf there is a GROUP BY clause, it must contain a list of expressions. Each expression will be referred to here as a \"key\". All the expressions in the SELECT, HAVING, and ORDER BY clauses must be calculated from keys or from aggregate functions. In other words, each column selected from the table must be used either in keys or inside aggregate functions.\n\nIf a query contains only table columns inside aggregate functions, the GROUP BY clause can be omitted, and aggregation by an empty set of keys is assumed.\n\nExample:\n```\nSELECT\n    count(),\n    median(FetchTiming > 60 ? 60 : FetchTiming),\n    count() - sum(Refresh)\nFROM hits\n```\nHowever, in contrast to standard SQL, if the table doesn't have any rows (either there aren't any at all, or there aren't any after using WHERE to filter), an empty result is returned, and not the result from one of the rows containing the initial values of aggregate functions.\n\nAs opposed to MySQL (and conforming to standard SQL), you can't get some value of some column that is not in a key or aggregate function (except constant expressions). To work around this, you can use the 'any' aggregate function (get the first encountered value) or 'min\/max'.\n\nExample:\n```\nSELECT\n    domainWithoutWWW(URL) AS domain,\n    count(),\n    any(Title) AS title -- getting the first occurred page header for each domain.\nFROM hits\nGROUP BY domain\n```\nFor every different key value encountered, GROUP BY calculates a set of aggregate function values.\n\nGROUP BY is not supported for array columns.\n\nA constant can't be specified as arguments for aggregate functions. Example: sum(1). Instead of this, you can get rid of the constant. Example: `count()`.\n\n#### NULL processing[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#null-processing \"Permanent link\")\nFor grouping, ClickHouse interprets [NULL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/syntax\/) as a value, and `NULL=NULL`.\n\nHere's an example to show what this means.\n\nAssume you have this table:\n```\n┌─x─┬────y─┐\n│ 1 │    2 │\n│ 2 │ ᴺᵁᴸᴸ │\n│ 3 │    2 │\n│ 3 │    3 │\n│ 3 │ ᴺᵁᴸᴸ │\n└───┴──────┘\n```\nThe query `SELECT sum(x), y FROM t_null_big GROUP BY y` results in:\n```\n┌─sum(x)─┬────y─┐\n│      4 │    2 │\n│      3 │    3 │\n│      5 │ ᴺᵁᴸᴸ │\n└────────┴──────┘\n```\nYou can see that `GROUP BY` for `У = NULL` summed up `x`, as if `NULL` is this value.\n\nIf you pass several keys to `GROUP BY`, the result will give you all the combinations of the selection, as if `NULL` were a specific value.\n\n#### WITH TOTALS Modifier[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#with-totals-modifier \"Permanent link\")\nIf the WITH TOTALS modifier is specified, another row will be calculated. This row will have key columns containing default values (zeros or empty lines), and columns of aggregate functions with the values calculated across all the rows (the \"total\" values).\n\nThis extra row is output in JSON\\*, TabSeparated\\*, and Pretty\\* formats, separately from the other rows. In the other formats, this row is not output.\n\nIn JSON\\* formats, this row is output as a separate 'totals' field. In TabSeparated\\* formats, the row comes after the main result, preceded by an empty row (after the other data). In Pretty\\* formats, the row is output as a separate table after the main result.\n\n`WITH TOTALS` can be run in different ways when HAVING is present. The behavior depends on the 'totals\\_mode' setting. By default, `totals_mode = 'before_having'`. In this case, 'totals' is calculated across all rows, including the ones that don't pass through HAVING and 'max\\_rows\\_to\\_group\\_by'.\n\nThe other alternatives include only the rows that pass through HAVING in 'totals', and behave differently with the setting `max_rows_to_group_by` and `group_by_overflow_mode = 'any'`.\n\n`after_having_exclusive` – Don't include rows that didn't pass through `max_rows_to_group_by`. In other words, 'totals' will have less than or the same number of rows as it would if `max_rows_to_group_by` were omitted.\n\n`after_having_inclusive` – Include all the rows that didn't pass through 'max\\_rows\\_to\\_group\\_by' in 'totals'. In other words, 'totals' will have more than or the same number of rows as it would if `max_rows_to_group_by` were omitted.\n\n`after_having_auto` – Count the number of rows that passed through HAVING. If it is more than a certain amount (by default, 50%), include all the rows that didn't pass through 'max\\_rows\\_to\\_group\\_by' in 'totals'. Otherwise, do not include them.\n\n`totals_auto_threshold` – By default, 0.5. The coefficient for `after_having_auto`.\n\nIf `max_rows_to_group_by` and `group_by_overflow_mode = 'any'` are not used, all variations of `after_having` are the same, and you can use any of them (for example, `after_having_auto`).\n\nYou can use WITH TOTALS in subqueries, including subqueries in the JOIN clause (in this case, the respective total values are combined).\n\n#### GROUP BY in External Memory[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-group-by-in-external-memory \"Permanent link\")\nYou can enable dumping temporary data to the disk to restrict memory usage during GROUP BY. The `max_bytes_before_external_group_by` setting determines the threshold RAM consumption for dumping GROUP BY temporary data to the file system. If set to 0 (the default), it is disabled.\n\nWhen using `max_bytes_before_external_group_by`, we recommend that you set max\\_memory\\_usage about twice as high. This is necessary because there are two stages to aggregation: reading the date and forming intermediate data (1) and merging the intermediate data (2). Dumping data to the file system can only occur during stage 1. If the temporary data wasn't dumped, then stage 2 might require up to the same amount of memory as in stage 1.\n\nFor example, if `max_memory_usage` was set to 10000000000 and you want to use external aggregation, it makes sense to set `max_bytes_before_external_group_by` to 10000000000, and max\\_memory\\_usage to 20000000000. When external aggregation is triggered (if there was at least one dump of temporary data), maximum consumption of RAM is only slightly more than `max_bytes_before_external_group_by`.\n\nWith distributed query processing, external aggregation is performed on remote servers. In order for the requestor server to use only a small amount of RAM, set `distributed_aggregation_memory_efficient` to 1.\n\nWhen merging data flushed to the disk, as well as when merging results from remote servers when the `distributed_aggregation_memory_efficient` setting is enabled, consumes up to 1\/256 \\* the number of threads from the total amount of RAM.\n\nWhen external aggregation is enabled, if there was less than `max_bytes_before_external_group_by` of data (i.e. data was not flushed), the query runs just as fast as without external aggregation. If any temporary data was flushed, the run time will be several times longer (approximately three times).\n\nIf you have an ORDER BY with a small LIMIT after GROUP BY, then the ORDER BY CLAUSE will not use significant amounts of RAM. But if the ORDER BY doesn't have LIMIT, don't forget to enable external sorting (`max_bytes_before_external_sort`).\n\n### LIMIT N BY Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#limit-n-by-clause \"Permanent link\")\nLIMIT N BY COLUMNS selects the top N rows for each group of COLUMNS. LIMIT N BY is not related to LIMIT; they can both be used in the same query. The key for LIMIT N BY can contain any number of columns or expressions.\n\nExample:\n```\nSELECT\n    domainWithoutWWW(URL) AS domain,\n    domainWithoutWWW(REFERRER_URL) AS referrer,\n    device_type,\n    count() cnt\nFROM hits\nGROUP BY domain, referrer, device_type\nORDER BY cnt DESC\nLIMIT 5 BY domain, device_type\nLIMIT 100\n```\nThe query will select the top 5 referrers for each `domain, device_type` pair, but not more than 100 rows (`LIMIT n BY + LIMIT`).\n\n### HAVING Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#having-clause \"Permanent link\")\nAllows filtering the result received after GROUP BY, similar to the WHERE clause. WHERE and HAVING differ in that WHERE is performed before aggregation (GROUP BY), while HAVING is performed after it. If aggregation is not performed, HAVING can't be used.\n\n### ORDER BY Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#order-by-clause \"Permanent link\")\nThe ORDER BY clause contains a list of expressions, which can each be assigned DESC or ASC (the sorting direction). If the direction is not specified, ASC is assumed. ASC is sorted in ascending order, and DESC in descending order. The sorting direction applies to a single expression, not to the entire list. Example: `ORDER BY Visits DESC, SearchPhrase`\n\nFor sorting by String values, you can specify collation (comparison). Example: `ORDER BY SearchPhrase COLLATE 'tr'` - for sorting by keyword in ascending order, using the Turkish alphabet, case insensitive, assuming that strings are UTF-8 encoded. COLLATE can be specified or not for each expression in ORDER BY independently. If ASC or DESC is specified, COLLATE is specified after it. When using COLLATE, sorting is always case-insensitive.\n\nWe only recommend using COLLATE for final sorting of a small number of rows, since sorting with COLLATE is less efficient than normal sorting by bytes.\n\nRows that have identical values for the list of sorting expressions are output in an arbitrary order, which can also be nondeterministic (different each time). If the ORDER BY clause is omitted, the order of the rows is also undefined, and may be nondeterministic as well.\n\n`NaN` and `NULL` sorting order:\n\n- With the modifier `NULLS FIRST` — First `NULL`, then `NaN`, then other values.\n- With the modifier `NULLS LAST` — First the values, then `NaN`, then `NULL`.\n- Default — The same as with the `NULLS LAST` modifier.\n\nExample:\n\nFor the table\n```\n┌─x─┬────y─┐\n│ 1 │ ᴺᵁᴸᴸ │\n│ 2 │    2 │\n│ 1 │  nan │\n│ 2 │    2 │\n│ 3 │    4 │\n│ 5 │    6 │\n│ 6 │  nan │\n│ 7 │ ᴺᵁᴸᴸ │\n│ 6 │    7 │\n│ 8 │    9 │\n└───┴──────┘\n```\nRun the query `SELECT * FROM t_null_nan ORDER BY y NULLS FIRST` to get:\n```\n┌─x─┬────y─┐\n│ 1 │ ᴺᵁᴸᴸ │\n│ 7 │ ᴺᵁᴸᴸ │\n│ 1 │  nan │\n│ 6 │  nan │\n│ 2 │    2 │\n│ 2 │    2 │\n│ 3 │    4 │\n│ 5 │    6 │\n│ 6 │    7 │\n│ 8 │    9 │\n└───┴──────┘\n```\nWhen floating point numbers are sorted, NaNs are separate from the other values. Regardless of the sorting order, NaNs come at the end. In other words, for ascending sorting they are placed as if they are larger than all the other numbers, while for descending sorting they are placed as if they are smaller than the rest.\n\nLess RAM is used if a small enough LIMIT is specified in addition to ORDER BY. Otherwise, the amount of memory spent is proportional to the volume of data for sorting. For distributed query processing, if GROUP BY is omitted, sorting is partially done on remote servers, and the results are merged on the requestor server. This means that for distributed sorting, the volume of data to sort can be greater than the amount of memory on a single server.\n\nIf there is not enough RAM, it is possible to perform sorting in external memory (creating temporary files on a disk). Use the setting `max_bytes_before_external_sort` for this purpose. If it is set to 0 (the default), external sorting is disabled. If it is enabled, when the volume of data to sort reaches the specified number of bytes, the collected data is sorted and dumped into a temporary file. After all data is read, all the sorted files are merged and the results are output. Files are written to the \/var\/lib\/clickhouse\/tmp\/ directory in the config (by default, but you can use the 'tmp\\_path' parameter to change this setting).\n\nRunning a query may use more memory than 'max\\_bytes\\_before\\_external\\_sort'. For this reason, this setting must have a value significantly smaller than 'max\\_memory\\_usage'. As an example, if your server has 128 GB of RAM and you need to run a single query, set 'max\\_memory\\_usage' to 100 GB, and 'max\\_bytes\\_before\\_external\\_sort' to 80 GB.\n\nExternal sorting works much less effectively than sorting in RAM.\n\n### SELECT Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-clause \"Permanent link\")\nThe expressions specified in the SELECT clause are analyzed after the calculations for all the clauses listed above are completed. More specifically, expressions are analyzed that are above the aggregate functions, if there are any aggregate functions. The aggregate functions and everything below them are calculated during aggregation (GROUP BY). These expressions work as if they are applied to separate rows in the result.\n\n### DISTINCT Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distinct \"Permanent link\")\nIf DISTINCT is specified, only a single row will remain out of all the sets of fully matching rows in the result. The result will be the same as if GROUP BY were specified across all the fields specified in SELECT without aggregate functions. But there are several differences from GROUP BY:\n\n- DISTINCT can be applied together with GROUP BY.\n- When ORDER BY is omitted and LIMIT is defined, the query stops running immediately after the required number of different rows has been read.\n- Data blocks are output as they are processed, without waiting for the entire query to finish running.\n\nDISTINCT is not supported if SELECT has at least one array column.\n\n`DISTINCT` works with [NULL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/syntax\/) as if `NULL` were a specific value, and `NULL=NULL`. In other words, in the `DISTINCT` results, different combinations with `NULL` only occur once.\n\n### LIMIT Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#limit-clause \"Permanent link\")\n`LIMIT m` allows you to select the first `m` rows from the result. `LIMIT n`, m allows you to select the first `m` rows from the result after skipping the first `n` rows.\n\n`n` and `m` must be non-negative integers.\n\nIf there isn't an ORDER BY clause that explicitly sorts results, the result may be arbitrary and nondeterministic.\n\n### UNION ALL Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#union-all-clause \"Permanent link\")\nYou can use UNION ALL to combine any number of queries. Example:\n```\nSELECT CounterID, 1 AS table, toInt64(count()) AS c\n    FROM test.hits\n    GROUP BY CounterID\n\nUNION ALL\n\nSELECT CounterID, 2 AS table, sum(Sign) AS c\n    FROM test.visits\n    GROUP BY CounterID\n    HAVING c > 0\n```\nOnly UNION ALL is supported. The regular UNION (UNION DISTINCT) is not supported. If you need UNION DISTINCT, you can write SELECT DISTINCT from a subquery containing UNION ALL.\n\nQueries that are parts of UNION ALL can be run simultaneously, and their results can be mixed together.\n\nThe structure of results (the number and type of columns) must match for the queries. But the column names can differ. In this case, the column names for the final result will be taken from the first query. Type casting is performed for unions. For example, if two queries being combined have the same field with non-`Nullable` and `Nullable` types from a compatible type, the resulting `UNION ALL` has a `Nullable` type field.\n\nQueries that are parts of UNION ALL can't be enclosed in brackets. ORDER BY and LIMIT are applied to separate queries, not to the final result. If you need to apply a conversion to the final result, you can put all the queries with UNION ALL in a subquery in the FROM clause.\n\n### INTO OUTFILE Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#into-outfile-clause \"Permanent link\")\nAdd the `INTO OUTFILE filename` clause (where filename is a string literal) to redirect query output to the specified file. In contrast to MySQL, the file is created on the client side. The query will fail if a file with the same filename already exists. This functionality is available in the command-line client and clickhouse-local (a query sent via HTTP interface will fail).\n\nThe default output format is TabSeparated (the same as in the command-line client batch mode).\n\n### FORMAT Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#format-clause \"Permanent link\")\nSpecify 'FORMAT format' to get data in any specified format. You can use this for convenience, or for creating dumps. For more information, see the section \"Formats\". If the FORMAT clause is omitted, the default format is used, which depends on both the settings and the interface used for accessing the DB. For the HTTP interface and the command-line client in batch mode, the default format is TabSeparated. For the command-line client in interactive mode, the default format is PrettyCompact (it has attractive and compact tables).\n\nWhen using the command-line client, data is passed to the client in an internal efficient format. The client independently interprets the FORMAT clause of the query and formats the data itself (thus relieving the network and the server from the load).\n\n### IN Operators[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-in-operators \"Permanent link\")\nThe `IN`, `NOT IN`, `GLOBAL IN`, and `GLOBAL NOT IN` operators are covered separately, since their functionality is quite rich.\n\nThe left side of the operator is either a single column or a tuple.\n\nExamples:\n```\nSELECT UserID IN (123, 456) FROM ...\nSELECT (CounterID, UserID) IN ((34, 123), (101500, 456)) FROM ...\n```\nIf the left side is a single column that is in the index, and the right side is a set of constants, the system uses the index for processing the query.\n\nDon't list too many values explicitly (i.e. millions). If a data set is large, put it in a temporary table (for example, see the section \"External data for query processing\"), then use a subquery.\n\nThe right side of the operator can be a set of constant expressions, a set of tuples with constant expressions (shown in the examples above), or the name of a database table or SELECT subquery in brackets.\n\nIf the right side of the operator is the name of a table (for example, `UserID IN users`), this is equivalent to the subquery `UserID IN (SELECT * FROM users)`. Use this when working with external data that is sent along with the query. For example, the query can be sent together with a set of user IDs loaded to the 'users' temporary table, which should be filtered.\n\nIf the right side of the operator is a table name that has the Set engine (a prepared data set that is always in RAM), the data set will not be created over again for each query.\n\nThe subquery may specify more than one column for filtering tuples. Example:\n```\nSELECT (CounterID, UserID) IN (SELECT CounterID, UserID FROM ...) FROM ...\n```\nThe columns to the left and right of the IN operator should have the same type.\n\nThe IN operator and subquery may occur in any part of the query, including in aggregate functions and lambda functions. Example:\n```\nSELECT\n    EventDate,\n    avg(UserID IN\n    (\n        SELECT UserID\n        FROM test.hits\n        WHERE EventDate = toDate('2014-03-17')\n    )) AS ratio\nFROM test.hits\nGROUP BY EventDate\nORDER BY EventDate ASC\n```\n```\n┌──EventDate─┬────ratio─┐\n│ 2014-03-17 │        1 │\n│ 2014-03-18 │ 0.807696 │\n│ 2014-03-19 │ 0.755406 │\n│ 2014-03-20 │ 0.723218 │\n│ 2014-03-21 │ 0.697021 │\n│ 2014-03-22 │ 0.647851 │\n│ 2014-03-23 │ 0.648416 │\n└────────────┴──────────┘\n```\nFor each day after March 17th, count the percentage of pageviews made by users who visited the site on March 17th. A subquery in the IN clause is always run just one time on a single server. There are no dependent subqueries.\n\n#### NULL processing[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#null-processing_1 \"Permanent link\")\nDuring request processing, the IN operator assumes that the result of an operation with [NULL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/syntax\/) is always equal to `0`, regardless of whether `NULL` is on the right or left side of the operator. `NULL` values are not included in any dataset, do not correspond to each other and cannot be compared.\n\nHere is an example with the `t_null` table:\n```\n┌─x─┬────y─┐\n│ 1 │ ᴺᵁᴸᴸ │\n│ 2 │    3 │\n└───┴──────┘\n```\nRunning the query `SELECT x FROM t_null WHERE y IN (NULL,3)` gives you the following result:\n```\n┌─x─┐\n│ 2 │\n└───┘\n```\nYou can see that the row in which `y = NULL` is thrown out of the query results. This is because ClickHouse can't decide whether `NULL` is included in the `(NULL,3)` set, returns `0` as the result of the operation, and `SELECT` excludes this row from the final output.\n```\nSELECT y IN (NULL, 3)\nFROM t_null\n\n┌─in(y, tuple(NULL, 3))─┐\n│                     0 │\n│                     1 │\n└───────────────────────┘\n```\n#### Distributed Subqueries[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distributed-subqueries \"Permanent link\")\nThere are two options for IN-s with subqueries (similar to JOINs): normal `IN` \/ `JOIN` and `GLOBAL IN` \/ `GLOBAL JOIN`. They differ in how they are run for distributed query processing.\n\nAttention\n\nRemember that the algorithms described below may work differently depending on the [settings](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/settings\/) `distributed_product_mode` setting.\n\nWhen using the regular IN, the query is sent to remote servers, and each of them runs the subqueries in the `IN` or `JOIN` clause.\n\nWhen using `GLOBAL IN` \/ `GLOBAL JOINs`, first all the subqueries are run for `GLOBAL IN` \/ `GLOBAL JOINs`, and the results are collected in temporary tables. Then the temporary tables are sent to each remote server, where the queries are run using this temporary data.\n\nFor a non-distributed query, use the regular `IN` \/ `JOIN`.\n\nBe careful when using subqueries in the `IN` \/ `JOIN` clauses for distributed query processing.\n\nLet's look at some examples. Assume that each server in the cluster has a normal **local\\_table**. Each server also has a **distributed\\_table** table with the **Distributed** type, which looks at all the servers in the cluster.\n\nFor a query to the **distributed\\_table**, the query will be sent to all the remote servers and run on them using the **local\\_table**.\n\nFor example, the query\n```\nSELECT uniq(UserID) FROM distributed_table\n```\nwill be sent to all remote servers as\n```\nSELECT uniq(UserID) FROM local_table\n```\nand run on each of them in parallel, until it reaches the stage where intermediate results can be combined. Then the intermediate results will be returned to the requestor server and merged on it, and the final result will be sent to the client.\n\nNow let's examine a query with IN:\n```\nSELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34)\n```\n- Calculation of the intersection of audiences of two sites.\n\nThis query will be sent to all remote servers as\n```\nSELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34)\n```\nIn other words, the data set in the IN clause will be collected on each server independently, only across the data that is stored locally on each of the servers.\n\nThis will work correctly and optimally if you are prepared for this case and have spread data across the cluster servers such that the data for a single UserID resides entirely on a single server. In this case, all the necessary data will be available locally on each server. Otherwise, the result will be inaccurate. We refer to this variation of the query as \"local IN\".\n\nTo correct how the query works when data is spread randomly across the cluster servers, you could specify **distributed\\_table** inside a subquery. The query would look like this:\n```\nSELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34)\n```\nThis query will be sent to all remote servers as\n```\nSELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34)\n```\nThe subquery will begin running on each remote server. Since the subquery uses a distributed table, the subquery that is on each remote server will be resent to every remote server as\n```\nSELECT UserID FROM local_table WHERE CounterID = 34\n```\nFor example, if you have a cluster of 100 servers, executing the entire query will require 10,000 elementary requests, which is generally considered unacceptable.\n\nIn such cases, you should always use GLOBAL IN instead of IN. Let's look at how it works for the query\n```\nSELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID GLOBAL IN (SELECT UserID FROM distributed_table WHERE CounterID = 34)\n```\nThe requestor server will run the subquery\n```\nSELECT UserID FROM distributed_table WHERE CounterID = 34\n```\nand the result will be put in a temporary table in RAM. Then the request will be sent to each remote server as\n```\nSELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID GLOBAL IN _data1\n```\nand the temporary table `_data1` will be sent to every remote server with the query (the name of the temporary table is implementation-defined).\n\nThis is more optimal than using the normal IN. However, keep the following points in mind:\n\n1. When creating a temporary table, data is not made unique. To reduce the volume of data transmitted over the network, specify DISTINCT in the subquery. (You don't need to do this for a normal IN.)\n2. The temporary table will be sent to all the remote servers. Transmission does not account for network topology. For example, if 10 remote servers reside in a datacenter that is very remote in relation to the requestor server, the data will be sent 10 times over the channel to the remote datacenter. Try to avoid large data sets when using GLOBAL IN.\n3. When transmitting data to remote servers, restrictions on network bandwidth are not configurable. You might overload the network.\n4. Try to distribute data across servers so that you don't need to use GLOBAL IN on a regular basis.\n5. If you need to use GLOBAL IN often, plan the location of the ClickHouse cluster so that a single group of replicas resides in no more than one data center with a fast network between them, so that a query can be processed entirely within a single data center.\n\nIt also makes sense to specify a local table in the `GLOBAL IN` clause, in case this local table is only available on the requestor server and you want to use data from it on remote servers.\n\n### Extreme Values[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#extreme-values \"Permanent link\")\nIn addition to results, you can also get minimum and maximum values for the results columns. To do this, set the **extremes** setting to 1. Minimums and maximums are calculated for numeric types, dates, and dates with times. For other columns, the default values are output.\n\nAn extra two rows are calculated – the minimums and maximums, respectively. These extra two rows are output in JSON\\*, TabSeparated\\*, and Pretty\\* formats, separate from the other rows. They are not output for other formats.\n\nIn JSON\\* formats, the extreme values are output in a separate 'extremes' field. In TabSeparated\\* formats, the row comes after the main result, and after 'totals' if present. It is preceded by an empty row (after the other data). In Pretty\\* formats, the row is output as a separate table after the main result, and after 'totals' if present.\n\nExtreme values are calculated for rows that have passed through LIMIT. However, when using 'LIMIT offset, size', the rows before 'offset' are included in 'extremes'. In stream requests, the result may also include a small number of rows that passed through LIMIT.\n\n### Notes[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#notes \"Permanent link\")\nThe `GROUP BY` and `ORDER BY` clauses do not support positional arguments. This contradicts MySQL, but conforms to standard SQL. For example, `GROUP BY 1, 2` will be interpreted as grouping by constants (i.e. aggregation of all rows into one).\n\nYou can use synonyms (`AS` aliases) in any part of a query.\n\nYou can put an asterisk in any part of a query instead of an expression. When the query is analyzed, the asterisk is expanded to a list of all table columns (excluding the `MATERIALIZED` and `ALIAS` columns). There are only a few cases when using an asterisk is justified:\n\n- When creating a table dump.\n- For tables containing just a few columns, such as system tables.\n- For getting information about what columns are in a table. In this case, set `LIMIT 1`. But it is better to use the `DESC TABLE` query.\n- When there is strong filtration on a small number of columns using `PREWHERE`.\n- In subqueries (since columns that aren't needed for the external query are excluded from subqueries).\n\nIn all other cases, we don't recommend using the asterisk, since it only gives you the drawbacks of a columnar DBMS instead of the advantages. In other words using the asterisk is not recommended.\n\n[Original article](https:\/\/clickhouse.yandex\/docs\/en\/query_language\/select\/)\n\n[Previous  Buffer](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/buffer\/ \"Buffer\")\n\n[Next  INSERT INTO](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/insert_into\/ \"INSERT INTO\")\n\n©2016–2019 Yandex LLC","attrs_readable_markdown":"## SELECT Queries Syntax[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-queries-syntax \"Permanent link\")\n`SELECT` performs data retrieval.\n```\nSELECT [DISTINCT] expr_list\n    [FROM [db.]table | (subquery) | table_function] [FINAL]\n    [SAMPLE sample_coeff]\n    [ARRAY JOIN ...]\n    [GLOBAL] [ANY|ALL] [INNER|LEFT|RIGHT|FULL|CROSS] [OUTER] JOIN (subquery)|table USING columns_list\n    [PREWHERE expr]\n    [WHERE expr]\n    [GROUP BY expr_list] [WITH TOTALS]\n    [HAVING expr]\n    [ORDER BY expr_list]\n    [LIMIT [n, ]m]\n    [UNION ALL ...]\n    [INTO OUTFILE filename]\n    [FORMAT format]\n    [LIMIT n BY columns]\n```\nAll the clauses are optional, except for the required list of expressions immediately after SELECT. The clauses below are described in almost the same order as in the query execution conveyor.\n\nIf the query omits the `DISTINCT`, `GROUP BY` and `ORDER BY` clauses and the `IN` and `JOIN` subqueries, the query will be completely stream processed, using O(1) amount of RAM. Otherwise, the query might consume a lot of RAM if the appropriate restrictions are not specified: `max_memory_usage`, `max_rows_to_group_by`, `max_rows_to_sort`, `max_rows_in_distinct`, `max_bytes_in_distinct`, `max_rows_in_set`, `max_bytes_in_set`, `max_rows_in_join`, `max_bytes_in_join`, `max_bytes_before_external_sort`, `max_bytes_before_external_group_by`. For more information, see the section \"Settings\". It is possible to use external sorting (saving temporary tables to a disk) and external aggregation. `The system does not have \"merge join\"`.\n\n### FROM Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#from-clause \"Permanent link\")\nIf the FROM clause is omitted, data will be read from the `system.one` table. The 'system.one' table contains exactly one row (this table fulfills the same purpose as the DUAL table found in other DBMSs).\n\nThe FROM clause specifies the table to read data from, or a subquery, or a table function; ARRAY JOIN and the regular JOIN may also be included (see below).\n\nInstead of a table, the SELECT subquery may be specified in brackets. In this case, the subquery processing pipeline will be built into the processing pipeline of an external query. In contrast to standard SQL, a synonym does not need to be specified after a subquery. For compatibility, it is possible to write 'AS name' after a subquery, but the specified name isn't used anywhere.\n\nA table function may be specified instead of a table. For more information, see the section \"Table functions\".\n\nTo execute a query, all the columns listed in the query are extracted from the appropriate table. Any columns not needed for the external query are thrown out of the subqueries. If a query does not list any columns (for example, SELECT count() FROM t), some column is extracted from the table anyway (the smallest one is preferred), in order to calculate the number of rows.\n\nThe FINAL modifier can be used only for a SELECT from a CollapsingMergeTree table. When you specify FINAL, data is selected fully \"collapsed\". Keep in mind that using FINAL leads to a selection that includes columns related to the primary key, in addition to the columns specified in the SELECT. Additionally, the query will be executed in a single stream, and data will be merged during query execution. This means that when using FINAL, the query is processed more slowly. In most cases, you should avoid using FINAL. For more information, see the section \"CollapsingMergeTree engine\".\n\n### SAMPLE Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-clause \"Permanent link\")\nThe `SAMPLE` clause allows for approximated query processing. Approximated query processing is only supported by the tables in the `MergeTree` family, and only if the sampling expression was specified during table creation (see [MergeTree engine](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/mergetree\/)).\n\nThe features of data sampling are listed below:\n\n- Data sampling is a deterministic mechanism. The result of the same `SELECT .. SAMPLE` query is always the same.\n- Sampling works consistently for different tables. For tables with a single sampling key, a sample with the same coefficient always selects the same subset of possible data. For example, a sample of user IDs takes rows with the same subset of all the possible user IDs from different tables. This allows using the sample in subqueries in the `IN` clause, as well as for manually correlating results of different queries with samples.\n- Sampling allows reading less data from a disk. Note that for this you must specify the sampling key correctly. For more details see [Creating a MergeTree Table](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/mergetree\/#table_engine-mergetree-creating-a-table).\n\nThe `SAMPLE` clause can be specified in several ways:\n\n- `SAMPLE k`, where `k` is a decimal number from 0 to 1. The query is executed on `k` percent of data. For example, `SAMPLE 0.1` runs the query on 10% of data. [Read more](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-k)\n- `SAMPLE n`, where `n` is a sufficiently large integer. The query is executed on a sample of at least `n` rows (but not significantly more than this). For example, `SAMPLE 10000000` runs the query on a minimum of 10,000,000 rows. [Read more](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-n)\n- `SAMPLE k OFFSET m` where `k` and `m` are numbers from 0 to 1. The query is executed on a sample of `k` percent of the data. The data used for the sample is offset by `m` percent. [Read more](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-offset)\n#### SAMPLE k[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-k \"Permanent link\")\nIn a `SAMPLE k` clause, `k` is a percent amount of data that the sample is taken from. The example is shown below:\n```\nSELECT\n    Title,\n    count() * 10 AS PageViews\nFROM hits_distributed\nSAMPLE 0.1\nWHERE\n    CounterID = 34\nGROUP BY Title\nORDER BY PageViews DESC LIMIT 1000\n```\nIn this example, the query is executed on a sample from 0.1 (10%) of data. Values of aggregate functions are not corrected automatically, so to get an approximate result, the value 'count()' is manually multiplied by 10.\n\n#### SAMPLE n[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-n \"Permanent link\")\nIn this case, the query is executed on a sample of at least `n` rows, where `n` is a sufficiently large integer. For example, `SAMPLE 10000000`.\n\nSince the minimum unit for data reading is one granule (its size is set by the `index_granularity` setting), it makes sense to set a sample that is much larger than the size of the granule.\n\nWhen using the `SAMPLE n` clause, the relative coefficient is calculated dynamically. Since you do not know which relative percent of data was processed, you do not know the coefficient the aggregate functions should be multiplied by (for example, you do not know if the `SAMPLE 1000000` was taken from a set of 10,000,000 rows or from a set of 1,000,000,000 rows). In this case, use the `_sample_factor` column to get the approximate result.\n\nThe `_sample_factor` is a virtual column that ClickHouse stores relative coefficients in. This column is created automatically when you create a table with the specified sampling key. The usage example is shown below:\n```\nSELECT sum(Duration * _sample_factor)\nFROM visits\nSAMPLE 10000000\n```\nIf you need to get the approximate count of rows in a `SELECT .. SAMPLE n` query, get the sum() of `_sample_factor` column instead of counting `count(column * _sample_factor)` value. For example:\n```\nSELECT sum(_sample_factor)\nFROM visits\nSAMPLE 10000000\n```\nNote that to calculate the average in a `SELECT .. SAMPLE n` query, you do not need to use `_sample_factor` column:\n```\nSELECT avg(Duration)\nFROM visits\nSAMPLE 10000000\n```\n#### SAMPLE k OFFSET m[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-sample-offset \"Permanent link\")\nYou can specify the `SAMPLE k OFFSET m` clause, where `k` and `m` are numbers from 0 to 1. Examples are shown below.\n\nExample 1.\n```\nSAMPLE 1\/10\n```\nIn this example, the sample is the 1\/10th of all data:\n\n`[++------------------]`\n\nExample 2.\n```\nSAMPLE 1\/10 OFFSET 1\/2\n```\nHere, the sample of 10% is taken from the second half of data.\n\n`[----------++--------]`\n\n### ARRAY JOIN Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-array-join-clause \"Permanent link\")\nAllows executing JOIN with an array or nested data structure. The intent is similar to the 'arrayJoin' function, but its functionality is broader.\n\n`ARRAY JOIN` is essentially `INNER JOIN` with an array. Example:\n```\n:) CREATE TABLE arrays_test (s String, arr Array(UInt8)) ENGINE = Memory\n\nCREATE TABLE arrays_test\n(\n    s String,\n    arr Array(UInt8)\n) ENGINE = Memory\n\nOk.\n\n0 rows in set. Elapsed: 0.001 sec.\n\n:) INSERT INTO arrays_test VALUES ('Hello', [1,2]), ('World', [3,4,5]), ('Goodbye', [])\n\nINSERT INTO arrays_test VALUES\n\nOk.\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT * FROM arrays_test\n\nSELECT *\nFROM arrays_test\n\n┌─s───────┬─arr─────┐\n│ Hello   │ [1,2]   │\n│ World   │ [3,4,5] │\n│ Goodbye │ []      │\n└─────────┴─────────┘\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT s, arr FROM arrays_test ARRAY JOIN arr\n\nSELECT s, arr\nFROM arrays_test\nARRAY JOIN arr\n\n┌─s─────┬─arr─┐\n│ Hello │   1 │\n│ Hello │   2 │\n│ World │   3 │\n│ World │   4 │\n│ World │   5 │\n└───────┴─────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nAn alias can be specified for an array in the ARRAY JOIN clause. In this case, an array item can be accessed by this alias, but the array itself by the original name. Example:\n```\n:) SELECT s, arr, a FROM arrays_test ARRAY JOIN arr AS a\n\nSELECT s, arr, a\nFROM arrays_test\nARRAY JOIN arr AS a\n\n┌─s─────┬─arr─────┬─a─┐\n│ Hello │ [1,2]   │ 1 │\n│ Hello │ [1,2]   │ 2 │\n│ World │ [3,4,5] │ 3 │\n│ World │ [3,4,5] │ 4 │\n│ World │ [3,4,5] │ 5 │\n└───────┴─────────┴───┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nMultiple arrays of the same size can be comma-separated in the ARRAY JOIN clause. In this case, JOIN is performed with them simultaneously (the direct sum, not the direct product). Example:\n```\n:) SELECT s, arr, a, num, mapped FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(x -> x + 1, arr) AS mapped\n\nSELECT s, arr, a, num, mapped\nFROM arrays_test\nARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(lambda(tuple(x), plus(x, 1)), arr) AS mapped\n\n┌─s─────┬─arr─────┬─a─┬─num─┬─mapped─┐\n│ Hello │ [1,2]   │ 1 │   1 │      2 │\n│ Hello │ [1,2]   │ 2 │   2 │      3 │\n│ World │ [3,4,5] │ 3 │   1 │      4 │\n│ World │ [3,4,5] │ 4 │   2 │      5 │\n│ World │ [3,4,5] │ 5 │   3 │      6 │\n└───────┴─────────┴───┴─────┴────────┘\n\n5 rows in set. Elapsed: 0.002 sec.\n\n:) SELECT s, arr, a, num, arrayEnumerate(arr) FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num\n\nSELECT s, arr, a, num, arrayEnumerate(arr)\nFROM arrays_test\nARRAY JOIN arr AS a, arrayEnumerate(arr) AS num\n\n┌─s─────┬─arr─────┬─a─┬─num─┬─arrayEnumerate(arr)─┐\n│ Hello │ [1,2]   │ 1 │   1 │ [1,2]               │\n│ Hello │ [1,2]   │ 2 │   2 │ [1,2]               │\n│ World │ [3,4,5] │ 3 │   1 │ [1,2,3]             │\n│ World │ [3,4,5] │ 4 │   2 │ [1,2,3]             │\n│ World │ [3,4,5] │ 5 │   3 │ [1,2,3]             │\n└───────┴─────────┴───┴─────┴─────────────────────┘\n\n5 rows in set. Elapsed: 0.002 sec.\n```\nARRAY JOIN also works with nested data structures. Example:\n```\n:) CREATE TABLE nested_test (s String, nest Nested(x UInt8, y UInt32)) ENGINE = Memory\n\nCREATE TABLE nested_test\n(\n    s String,\n    nest Nested(\n    x UInt8,\n    y UInt32)\n) ENGINE = Memory\n\nOk.\n\n0 rows in set. Elapsed: 0.006 sec.\n\n:) INSERT INTO nested_test VALUES ('Hello', [1,2], [10,20]), ('World', [3,4,5], [30,40,50]), ('Goodbye', [], [])\n\nINSERT INTO nested_test VALUES\n\nOk.\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT * FROM nested_test\n\nSELECT *\nFROM nested_test\n\n┌─s───────┬─nest.x──┬─nest.y─────┐\n│ Hello   │ [1,2]   │ [10,20]    │\n│ World   │ [3,4,5] │ [30,40,50] │\n│ Goodbye │ []      │ []         │\n└─────────┴─────────┴────────────┘\n\n3 rows in set. Elapsed: 0.001 sec.\n\n:) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest\n\nSELECT s, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN nest\n\n┌─s─────┬─nest.x─┬─nest.y─┐\n│ Hello │      1 │     10 │\n│ Hello │      2 │     20 │\n│ World │      3 │     30 │\n│ World │      4 │     40 │\n│ World │      5 │     50 │\n└───────┴────────┴────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nWhen specifying names of nested data structures in ARRAY JOIN, the meaning is the same as ARRAY JOIN with all the array elements that it consists of. Example:\n```\n:) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x, nest.y\n\nSELECT s, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN `nest.x`, `nest.y`\n\n┌─s─────┬─nest.x─┬─nest.y─┐\n│ Hello │      1 │     10 │\n│ Hello │      2 │     20 │\n│ World │      3 │     30 │\n│ World │      4 │     40 │\n│ World │      5 │     50 │\n└───────┴────────┴────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nThis variation also makes sense:\n```\n:) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x\n\nSELECT s, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN `nest.x`\n\n┌─s─────┬─nest.x─┬─nest.y─────┐\n│ Hello │      1 │ [10,20]    │\n│ Hello │      2 │ [10,20]    │\n│ World │      3 │ [30,40,50] │\n│ World │      4 │ [30,40,50] │\n│ World │      5 │ [30,40,50] │\n└───────┴────────┴────────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nAn alias may be used for a nested data structure, in order to select either the JOIN result or the source array. Example:\n```\n:) SELECT s, n.x, n.y, nest.x, nest.y FROM nested_test ARRAY JOIN nest AS n\n\nSELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`\nFROM nested_test\nARRAY JOIN nest AS n\n\n┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┐\n│ Hello │   1 │  10 │ [1,2]   │ [10,20]    │\n│ Hello │   2 │  20 │ [1,2]   │ [10,20]    │\n│ World │   3 │  30 │ [3,4,5] │ [30,40,50] │\n│ World │   4 │  40 │ [3,4,5] │ [30,40,50] │\n│ World │   5 │  50 │ [3,4,5] │ [30,40,50] │\n└───────┴─────┴─────┴─────────┴────────────┘\n\n5 rows in set. Elapsed: 0.001 sec.\n```\nExample of using the arrayEnumerate function:\n```\n:) SELECT s, n.x, n.y, nest.x, nest.y, num FROM nested_test ARRAY JOIN nest AS n, arrayEnumerate(nest.x) AS num\n\nSELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`, num\nFROM nested_test\nARRAY JOIN nest AS n, arrayEnumerate(`nest.x`) AS num\n\n┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┬─num─┐\n│ Hello │   1 │  10 │ [1,2]   │ [10,20]    │   1 │\n│ Hello │   2 │  20 │ [1,2]   │ [10,20]    │   2 │\n│ World │   3 │  30 │ [3,4,5] │ [30,40,50] │   1 │\n│ World │   4 │  40 │ [3,4,5] │ [30,40,50] │   2 │\n│ World │   5 │  50 │ [3,4,5] │ [30,40,50] │   3 │\n└───────┴─────┴─────┴─────────┴────────────┴─────┘\n\n5 rows in set. Elapsed: 0.002 sec.\n```\nThe query can only specify a single ARRAY JOIN clause.\n\nThe corresponding conversion can be performed before the WHERE\/PREWHERE clause (if its result is needed in this clause), or after completing WHERE\/PREWHERE (to reduce the volume of calculations).\n\n### JOIN Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-join \"Permanent link\")\nJoins the data in the normal [SQL JOIN](https:\/\/en.wikipedia.org\/wiki\/Join_\\(SQL\\)) sense.\n\nNote\n\nNot related to [ARRAY JOIN](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-array-join-clause).\n```\nSELECT <expr_list>\nFROM <left_subquery>\n[GLOBAL] [ANY|ALL] [INNER|LEFT|RIGHT|FULL|CROSS] [OUTER] JOIN <right_subquery>\n(ON <expr_list>)|(USING <column_list>) ...\n```\nThe table names can be specified instead of `<left_subquery>` and `<right_subquery>`. This is equivalent to the `SELECT * FROM table` subquery, except in a special case when the table has the [Join](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/join\/) engine – an array prepared for joining.\n\n**Supported types of `JOIN`**\n\n- `INNER JOIN` (or `JOIN`)\n- `LEFT JOIN` (or `LEFT OUTER JOIN`)\n- `RIGHT JOIN` (or `RIGHT OUTER JOIN`)\n- `FULL JOIN` (or `FULL OUTER JOIN`)\n- `CROSS JOIN` (or `,` )\n\nSee standard [SQL JOIN](https:\/\/en.wikipedia.org\/wiki\/Join_\\(SQL\\)) description.\n\n**ANY or ALL strictness**\n\nIf `ALL` is specified and the right table has several matching rows, the data will be multiplied by the number of these rows. This is the normal `JOIN` behavior for standard SQL. If `ANY` is specified and the right table has several matching rows, only the first one found is joined. If the right table has only one matching row, the results of `ANY` and `ALL` are the same.\n\nTo set the default strictness value, use the session configuration parameter [join\\_default\\_strictness](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/settings\/#settings-join_default_strictness).\n\n**GLOBAL JOIN**\n\nWhen using a normal `JOIN`, the query is sent to remote servers. Subqueries are run on each of them in order to make the right table, and the join is performed with this table. In other words, the right table is formed on each server separately.\n\nWhen using `GLOBAL ... JOIN`, first the requestor server runs a subquery to calculate the right table. This temporary table is passed to each remote server, and queries are run on them using the temporary data that was transmitted.\n\nBe careful when using `GLOBAL`. For more information, see the section [Distributed subqueries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distributed-subqueries).\n\n**Usage Recommendations**\n\nAll columns that are not needed for the `JOIN` are deleted from the subquery.\n\nWhen running a `JOIN`, there is no optimization of the order of execution in relation to other stages of the query. The join (a search in the right table) is run before filtering in `WHERE` and before aggregation. In order to explicitly set the processing order, we recommend running a `JOIN` subquery with a subquery.\n\nExample:\n```\nSELECT\n    CounterID,\n    hits,\n    visits\nFROM\n(\n    SELECT\n        CounterID,\n        count() AS hits\n    FROM test.hits\n    GROUP BY CounterID\n) ANY LEFT JOIN\n(\n    SELECT\n        CounterID,\n        sum(Sign) AS visits\n    FROM test.visits\n    GROUP BY CounterID\n) USING CounterID\nORDER BY hits DESC\nLIMIT 10\n```\n```\n┌─CounterID─┬───hits─┬─visits─┐\n│   1143050 │ 523264 │  13665 │\n│    731962 │ 475698 │ 102716 │\n│    722545 │ 337212 │ 108187 │\n│    722889 │ 252197 │  10547 │\n│   2237260 │ 196036 │   9522 │\n│  23057320 │ 147211 │   7689 │\n│    722818 │  90109 │  17847 │\n│     48221 │  85379 │   4652 │\n│  19762435 │  77807 │   7026 │\n│    722884 │  77492 │  11056 │\n└───────────┴────────┴────────┘\n```\nSubqueries don't allow you to set names or use them for referencing a column from a specific subquery. The columns specified in `USING` must have the same names in both subqueries, and the other columns must be named differently. You can use aliases to change the names of columns in subqueries (the example uses the aliases 'hits' and 'visits').\n\nThe `USING` clause specifies one or more columns to join, which establishes the equality of these columns. The list of columns is set without brackets. More complex join conditions are not supported.\n\nThe right table (the subquery result) resides in RAM. If there isn't enough memory, you can't run a `JOIN`.\n\nEach time a query is run with the same `JOIN`, the subquery is run again because the result is not cached. To avoid this, use the special [Join](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/table_engines\/join\/) table engine, which is a prepared array for joining that is always in RAM.\n\nIn some cases, it is more efficient to use `IN` instead of `JOIN`. Among the various types of `JOIN`, the most efficient is `ANY LEFT JOIN`, then `ANY INNER JOIN`. The least efficient are `ALL LEFT JOIN` and `ALL INNER JOIN`.\n\nIf you need a `JOIN` for joining with dimension tables (these are relatively small tables that contain dimension properties, such as names for advertising campaigns), a `JOIN` might not be very convenient due to the bulky syntax and the fact that the right table is re-accessed for every query. For such cases, there is an \"external dictionaries\" feature that you should use instead of `JOIN`. For more information, see the section [External dictionaries](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/dicts\/external_dicts\/).\n\n#### Processing of Empty or NULL Cells[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#processing-of-empty-or-null-cells \"Permanent link\")\nWhile joining tables, the empty cells may appear. The setting [join\\_use\\_nulls](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/settings\/#settings-join_use_nulls) define how ClickHouse fills these cells.\n\nIf the `JOIN` keys are [Nullable](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/data_types\/nullable\/) fields, the rows where at least one of the keys has the value [NULL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/syntax\/#null-literal) are not joined.\n\n### WHERE Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#where-clause \"Permanent link\")\nIf there is a WHERE clause, it must contain an expression with the UInt8 type. This is usually an expression with comparison and logical operators. This expression will be used for filtering data before all other transformations.\n\nIf indexes are supported by the database table engine, the expression is evaluated on the ability to use indexes.\n\n### PREWHERE Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#prewhere-clause \"Permanent link\")\nThis clause has the same meaning as the WHERE clause. The difference is in which data is read from the table. When using PREWHERE, first only the columns necessary for executing PREWHERE are read. Then the other columns are read that are needed for running the query, but only those blocks where the PREWHERE expression is true.\n\nIt makes sense to use PREWHERE if there are filtration conditions that are used by a minority of the columns in the query, but that provide strong data filtration. This reduces the volume of data to read.\n\nFor example, it is useful to write PREWHERE for queries that extract a large number of columns, but that only have filtration for a few columns.\n\nPREWHERE is only supported by tables from the `*MergeTree` family.\n\nA query may simultaneously specify PREWHERE and WHERE. In this case, PREWHERE precedes WHERE.\n\nIf the 'optimize\\_move\\_to\\_prewhere' setting is set to 1 and PREWHERE is omitted, the system uses heuristics to automatically move parts of expressions from WHERE to PREWHERE.\n\n### GROUP BY Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-group-by-clause \"Permanent link\")\nThis is one of the most important parts of a column-oriented DBMS.\n\nIf there is a GROUP BY clause, it must contain a list of expressions. Each expression will be referred to here as a \"key\". All the expressions in the SELECT, HAVING, and ORDER BY clauses must be calculated from keys or from aggregate functions. In other words, each column selected from the table must be used either in keys or inside aggregate functions.\n\nIf a query contains only table columns inside aggregate functions, the GROUP BY clause can be omitted, and aggregation by an empty set of keys is assumed.\n\nExample:\n```\nSELECT\n    count(),\n    median(FetchTiming > 60 ? 60 : FetchTiming),\n    count() - sum(Refresh)\nFROM hits\n```\nHowever, in contrast to standard SQL, if the table doesn't have any rows (either there aren't any at all, or there aren't any after using WHERE to filter), an empty result is returned, and not the result from one of the rows containing the initial values of aggregate functions.\n\nAs opposed to MySQL (and conforming to standard SQL), you can't get some value of some column that is not in a key or aggregate function (except constant expressions). To work around this, you can use the 'any' aggregate function (get the first encountered value) or 'min\/max'.\n\nExample:\n```\nSELECT\n    domainWithoutWWW(URL) AS domain,\n    count(),\n    any(Title) AS title -- getting the first occurred page header for each domain.\nFROM hits\nGROUP BY domain\n```\nFor every different key value encountered, GROUP BY calculates a set of aggregate function values.\n\nGROUP BY is not supported for array columns.\n\nA constant can't be specified as arguments for aggregate functions. Example: sum(1). Instead of this, you can get rid of the constant. Example: `count()`.\n\n#### NULL processing[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#null-processing \"Permanent link\")\nFor grouping, ClickHouse interprets [NULL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/syntax\/) as a value, and `NULL=NULL`.\n\nHere's an example to show what this means.\n\nAssume you have this table:\n```\n┌─x─┬────y─┐\n│ 1 │    2 │\n│ 2 │ ᴺᵁᴸᴸ │\n│ 3 │    2 │\n│ 3 │    3 │\n│ 3 │ ᴺᵁᴸᴸ │\n└───┴──────┘\n```\nThe query `SELECT sum(x), y FROM t_null_big GROUP BY y` results in:\n```\n┌─sum(x)─┬────y─┐\n│      4 │    2 │\n│      3 │    3 │\n│      5 │ ᴺᵁᴸᴸ │\n└────────┴──────┘\n```\nYou can see that `GROUP BY` for `У = NULL` summed up `x`, as if `NULL` is this value.\n\nIf you pass several keys to `GROUP BY`, the result will give you all the combinations of the selection, as if `NULL` were a specific value.\n\n#### WITH TOTALS Modifier[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#with-totals-modifier \"Permanent link\")\nIf the WITH TOTALS modifier is specified, another row will be calculated. This row will have key columns containing default values (zeros or empty lines), and columns of aggregate functions with the values calculated across all the rows (the \"total\" values).\n\nThis extra row is output in JSON\\*, TabSeparated\\*, and Pretty\\* formats, separately from the other rows. In the other formats, this row is not output.\n\nIn JSON\\* formats, this row is output as a separate 'totals' field. In TabSeparated\\* formats, the row comes after the main result, preceded by an empty row (after the other data). In Pretty\\* formats, the row is output as a separate table after the main result.\n\n`WITH TOTALS` can be run in different ways when HAVING is present. The behavior depends on the 'totals\\_mode' setting. By default, `totals_mode = 'before_having'`. In this case, 'totals' is calculated across all rows, including the ones that don't pass through HAVING and 'max\\_rows\\_to\\_group\\_by'.\n\nThe other alternatives include only the rows that pass through HAVING in 'totals', and behave differently with the setting `max_rows_to_group_by` and `group_by_overflow_mode = 'any'`.\n\n`after_having_exclusive` – Don't include rows that didn't pass through `max_rows_to_group_by`. In other words, 'totals' will have less than or the same number of rows as it would if `max_rows_to_group_by` were omitted.\n\n`after_having_inclusive` – Include all the rows that didn't pass through 'max\\_rows\\_to\\_group\\_by' in 'totals'. In other words, 'totals' will have more than or the same number of rows as it would if `max_rows_to_group_by` were omitted.\n\n`after_having_auto` – Count the number of rows that passed through HAVING. If it is more than a certain amount (by default, 50%), include all the rows that didn't pass through 'max\\_rows\\_to\\_group\\_by' in 'totals'. Otherwise, do not include them.\n\n`totals_auto_threshold` – By default, 0.5. The coefficient for `after_having_auto`.\n\nIf `max_rows_to_group_by` and `group_by_overflow_mode = 'any'` are not used, all variations of `after_having` are the same, and you can use any of them (for example, `after_having_auto`).\n\nYou can use WITH TOTALS in subqueries, including subqueries in the JOIN clause (in this case, the respective total values are combined).\n\n#### GROUP BY in External Memory[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-group-by-in-external-memory \"Permanent link\")\nYou can enable dumping temporary data to the disk to restrict memory usage during GROUP BY. The `max_bytes_before_external_group_by` setting determines the threshold RAM consumption for dumping GROUP BY temporary data to the file system. If set to 0 (the default), it is disabled.\n\nWhen using `max_bytes_before_external_group_by`, we recommend that you set max\\_memory\\_usage about twice as high. This is necessary because there are two stages to aggregation: reading the date and forming intermediate data (1) and merging the intermediate data (2). Dumping data to the file system can only occur during stage 1. If the temporary data wasn't dumped, then stage 2 might require up to the same amount of memory as in stage 1.\n\nFor example, if `max_memory_usage` was set to 10000000000 and you want to use external aggregation, it makes sense to set `max_bytes_before_external_group_by` to 10000000000, and max\\_memory\\_usage to 20000000000. When external aggregation is triggered (if there was at least one dump of temporary data), maximum consumption of RAM is only slightly more than `max_bytes_before_external_group_by`.\n\nWith distributed query processing, external aggregation is performed on remote servers. In order for the requestor server to use only a small amount of RAM, set `distributed_aggregation_memory_efficient` to 1.\n\nWhen merging data flushed to the disk, as well as when merging results from remote servers when the `distributed_aggregation_memory_efficient` setting is enabled, consumes up to 1\/256 \\* the number of threads from the total amount of RAM.\n\nWhen external aggregation is enabled, if there was less than `max_bytes_before_external_group_by` of data (i.e. data was not flushed), the query runs just as fast as without external aggregation. If any temporary data was flushed, the run time will be several times longer (approximately three times).\n\nIf you have an ORDER BY with a small LIMIT after GROUP BY, then the ORDER BY CLAUSE will not use significant amounts of RAM. But if the ORDER BY doesn't have LIMIT, don't forget to enable external sorting (`max_bytes_before_external_sort`).\n\n### LIMIT N BY Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#limit-n-by-clause \"Permanent link\")\nLIMIT N BY COLUMNS selects the top N rows for each group of COLUMNS. LIMIT N BY is not related to LIMIT; they can both be used in the same query. The key for LIMIT N BY can contain any number of columns or expressions.\n\nExample:\n```\nSELECT\n    domainWithoutWWW(URL) AS domain,\n    domainWithoutWWW(REFERRER_URL) AS referrer,\n    device_type,\n    count() cnt\nFROM hits\nGROUP BY domain, referrer, device_type\nORDER BY cnt DESC\nLIMIT 5 BY domain, device_type\nLIMIT 100\n```\nThe query will select the top 5 referrers for each `domain, device_type` pair, but not more than 100 rows (`LIMIT n BY + LIMIT`).\n\n### HAVING Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#having-clause \"Permanent link\")\nAllows filtering the result received after GROUP BY, similar to the WHERE clause. WHERE and HAVING differ in that WHERE is performed before aggregation (GROUP BY), while HAVING is performed after it. If aggregation is not performed, HAVING can't be used.\n\n### ORDER BY Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#order-by-clause \"Permanent link\")\nThe ORDER BY clause contains a list of expressions, which can each be assigned DESC or ASC (the sorting direction). If the direction is not specified, ASC is assumed. ASC is sorted in ascending order, and DESC in descending order. The sorting direction applies to a single expression, not to the entire list. Example: `ORDER BY Visits DESC, SearchPhrase`\n\nFor sorting by String values, you can specify collation (comparison). Example: `ORDER BY SearchPhrase COLLATE 'tr'` - for sorting by keyword in ascending order, using the Turkish alphabet, case insensitive, assuming that strings are UTF-8 encoded. COLLATE can be specified or not for each expression in ORDER BY independently. If ASC or DESC is specified, COLLATE is specified after it. When using COLLATE, sorting is always case-insensitive.\n\nWe only recommend using COLLATE for final sorting of a small number of rows, since sorting with COLLATE is less efficient than normal sorting by bytes.\n\nRows that have identical values for the list of sorting expressions are output in an arbitrary order, which can also be nondeterministic (different each time). If the ORDER BY clause is omitted, the order of the rows is also undefined, and may be nondeterministic as well.\n\n`NaN` and `NULL` sorting order:\n\n- With the modifier `NULLS FIRST` — First `NULL`, then `NaN`, then other values.\n- With the modifier `NULLS LAST` — First the values, then `NaN`, then `NULL`.\n- Default — The same as with the `NULLS LAST` modifier.\n\nExample:\n\nFor the table\n```\n┌─x─┬────y─┐\n│ 1 │ ᴺᵁᴸᴸ │\n│ 2 │    2 │\n│ 1 │  nan │\n│ 2 │    2 │\n│ 3 │    4 │\n│ 5 │    6 │\n│ 6 │  nan │\n│ 7 │ ᴺᵁᴸᴸ │\n│ 6 │    7 │\n│ 8 │    9 │\n└───┴──────┘\n```\nRun the query `SELECT * FROM t_null_nan ORDER BY y NULLS FIRST` to get:\n```\n┌─x─┬────y─┐\n│ 1 │ ᴺᵁᴸᴸ │\n│ 7 │ ᴺᵁᴸᴸ │\n│ 1 │  nan │\n│ 6 │  nan │\n│ 2 │    2 │\n│ 2 │    2 │\n│ 3 │    4 │\n│ 5 │    6 │\n│ 6 │    7 │\n│ 8 │    9 │\n└───┴──────┘\n```\nWhen floating point numbers are sorted, NaNs are separate from the other values. Regardless of the sorting order, NaNs come at the end. In other words, for ascending sorting they are placed as if they are larger than all the other numbers, while for descending sorting they are placed as if they are smaller than the rest.\n\nLess RAM is used if a small enough LIMIT is specified in addition to ORDER BY. Otherwise, the amount of memory spent is proportional to the volume of data for sorting. For distributed query processing, if GROUP BY is omitted, sorting is partially done on remote servers, and the results are merged on the requestor server. This means that for distributed sorting, the volume of data to sort can be greater than the amount of memory on a single server.\n\nIf there is not enough RAM, it is possible to perform sorting in external memory (creating temporary files on a disk). Use the setting `max_bytes_before_external_sort` for this purpose. If it is set to 0 (the default), external sorting is disabled. If it is enabled, when the volume of data to sort reaches the specified number of bytes, the collected data is sorted and dumped into a temporary file. After all data is read, all the sorted files are merged and the results are output. Files are written to the \/var\/lib\/clickhouse\/tmp\/ directory in the config (by default, but you can use the 'tmp\\_path' parameter to change this setting).\n\nRunning a query may use more memory than 'max\\_bytes\\_before\\_external\\_sort'. For this reason, this setting must have a value significantly smaller than 'max\\_memory\\_usage'. As an example, if your server has 128 GB of RAM and you need to run a single query, set 'max\\_memory\\_usage' to 100 GB, and 'max\\_bytes\\_before\\_external\\_sort' to 80 GB.\n\nExternal sorting works much less effectively than sorting in RAM.\n\n### SELECT Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-clause \"Permanent link\")\nThe expressions specified in the SELECT clause are analyzed after the calculations for all the clauses listed above are completed. More specifically, expressions are analyzed that are above the aggregate functions, if there are any aggregate functions. The aggregate functions and everything below them are calculated during aggregation (GROUP BY). These expressions work as if they are applied to separate rows in the result.\n\n### DISTINCT Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distinct \"Permanent link\")\nIf DISTINCT is specified, only a single row will remain out of all the sets of fully matching rows in the result. The result will be the same as if GROUP BY were specified across all the fields specified in SELECT without aggregate functions. But there are several differences from GROUP BY:\n\n- DISTINCT can be applied together with GROUP BY.\n- When ORDER BY is omitted and LIMIT is defined, the query stops running immediately after the required number of different rows has been read.\n- Data blocks are output as they are processed, without waiting for the entire query to finish running.\n\nDISTINCT is not supported if SELECT has at least one array column.\n\n`DISTINCT` works with [NULL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/syntax\/) as if `NULL` were a specific value, and `NULL=NULL`. In other words, in the `DISTINCT` results, different combinations with `NULL` only occur once.\n\n### LIMIT Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#limit-clause \"Permanent link\")\n`LIMIT m` allows you to select the first `m` rows from the result. `LIMIT n`, m allows you to select the first `m` rows from the result after skipping the first `n` rows.\n\n`n` and `m` must be non-negative integers.\n\nIf there isn't an ORDER BY clause that explicitly sorts results, the result may be arbitrary and nondeterministic.\n\n### UNION ALL Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#union-all-clause \"Permanent link\")\nYou can use UNION ALL to combine any number of queries. Example:\n```\nSELECT CounterID, 1 AS table, toInt64(count()) AS c\n    FROM test.hits\n    GROUP BY CounterID\n\nUNION ALL\n\nSELECT CounterID, 2 AS table, sum(Sign) AS c\n    FROM test.visits\n    GROUP BY CounterID\n    HAVING c > 0\n```\nOnly UNION ALL is supported. The regular UNION (UNION DISTINCT) is not supported. If you need UNION DISTINCT, you can write SELECT DISTINCT from a subquery containing UNION ALL.\n\nQueries that are parts of UNION ALL can be run simultaneously, and their results can be mixed together.\n\nThe structure of results (the number and type of columns) must match for the queries. But the column names can differ. In this case, the column names for the final result will be taken from the first query. Type casting is performed for unions. For example, if two queries being combined have the same field with non-`Nullable` and `Nullable` types from a compatible type, the resulting `UNION ALL` has a `Nullable` type field.\n\nQueries that are parts of UNION ALL can't be enclosed in brackets. ORDER BY and LIMIT are applied to separate queries, not to the final result. If you need to apply a conversion to the final result, you can put all the queries with UNION ALL in a subquery in the FROM clause.\n\n### INTO OUTFILE Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#into-outfile-clause \"Permanent link\")\nAdd the `INTO OUTFILE filename` clause (where filename is a string literal) to redirect query output to the specified file. In contrast to MySQL, the file is created on the client side. The query will fail if a file with the same filename already exists. This functionality is available in the command-line client and clickhouse-local (a query sent via HTTP interface will fail).\n\nThe default output format is TabSeparated (the same as in the command-line client batch mode).\n\n### FORMAT Clause[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#format-clause \"Permanent link\")\nSpecify 'FORMAT format' to get data in any specified format. You can use this for convenience, or for creating dumps. For more information, see the section \"Formats\". If the FORMAT clause is omitted, the default format is used, which depends on both the settings and the interface used for accessing the DB. For the HTTP interface and the command-line client in batch mode, the default format is TabSeparated. For the command-line client in interactive mode, the default format is PrettyCompact (it has attractive and compact tables).\n\nWhen using the command-line client, data is passed to the client in an internal efficient format. The client independently interprets the FORMAT clause of the query and formats the data itself (thus relieving the network and the server from the load).\n\n### IN Operators[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-in-operators \"Permanent link\")\nThe `IN`, `NOT IN`, `GLOBAL IN`, and `GLOBAL NOT IN` operators are covered separately, since their functionality is quite rich.\n\nThe left side of the operator is either a single column or a tuple.\n\nExamples:\n```\nSELECT UserID IN (123, 456) FROM ...\nSELECT (CounterID, UserID) IN ((34, 123), (101500, 456)) FROM ...\n```\nIf the left side is a single column that is in the index, and the right side is a set of constants, the system uses the index for processing the query.\n\nDon't list too many values explicitly (i.e. millions). If a data set is large, put it in a temporary table (for example, see the section \"External data for query processing\"), then use a subquery.\n\nThe right side of the operator can be a set of constant expressions, a set of tuples with constant expressions (shown in the examples above), or the name of a database table or SELECT subquery in brackets.\n\nIf the right side of the operator is the name of a table (for example, `UserID IN users`), this is equivalent to the subquery `UserID IN (SELECT * FROM users)`. Use this when working with external data that is sent along with the query. For example, the query can be sent together with a set of user IDs loaded to the 'users' temporary table, which should be filtered.\n\nIf the right side of the operator is a table name that has the Set engine (a prepared data set that is always in RAM), the data set will not be created over again for each query.\n\nThe subquery may specify more than one column for filtering tuples. Example:\n```\nSELECT (CounterID, UserID) IN (SELECT CounterID, UserID FROM ...) FROM ...\n```\nThe columns to the left and right of the IN operator should have the same type.\n\nThe IN operator and subquery may occur in any part of the query, including in aggregate functions and lambda functions. Example:\n```\nSELECT\n    EventDate,\n    avg(UserID IN\n    (\n        SELECT UserID\n        FROM test.hits\n        WHERE EventDate = toDate('2014-03-17')\n    )) AS ratio\nFROM test.hits\nGROUP BY EventDate\nORDER BY EventDate ASC\n```\n```\n┌──EventDate─┬────ratio─┐\n│ 2014-03-17 │        1 │\n│ 2014-03-18 │ 0.807696 │\n│ 2014-03-19 │ 0.755406 │\n│ 2014-03-20 │ 0.723218 │\n│ 2014-03-21 │ 0.697021 │\n│ 2014-03-22 │ 0.647851 │\n│ 2014-03-23 │ 0.648416 │\n└────────────┴──────────┘\n```\nFor each day after March 17th, count the percentage of pageviews made by users who visited the site on March 17th. A subquery in the IN clause is always run just one time on a single server. There are no dependent subqueries.\n\n#### NULL processing[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#null-processing_1 \"Permanent link\")\nDuring request processing, the IN operator assumes that the result of an operation with [NULL](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/syntax\/) is always equal to `0`, regardless of whether `NULL` is on the right or left side of the operator. `NULL` values are not included in any dataset, do not correspond to each other and cannot be compared.\n\nHere is an example with the `t_null` table:\n```\n┌─x─┬────y─┐\n│ 1 │ ᴺᵁᴸᴸ │\n│ 2 │    3 │\n└───┴──────┘\n```\nRunning the query `SELECT x FROM t_null WHERE y IN (NULL,3)` gives you the following result:\n```\n┌─x─┐\n│ 2 │\n└───┘\n```\nYou can see that the row in which `y = NULL` is thrown out of the query results. This is because ClickHouse can't decide whether `NULL` is included in the `(NULL,3)` set, returns `0` as the result of the operation, and `SELECT` excludes this row from the final output.\n```\nSELECT y IN (NULL, 3)\nFROM t_null\n\n┌─in(y, tuple(NULL, 3))─┐\n│                     0 │\n│                     1 │\n└───────────────────────┘\n```\n#### Distributed Subqueries[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#select-distributed-subqueries \"Permanent link\")\nThere are two options for IN-s with subqueries (similar to JOINs): normal `IN` \/ `JOIN` and `GLOBAL IN` \/ `GLOBAL JOIN`. They differ in how they are run for distributed query processing.\n\nAttention\n\nRemember that the algorithms described below may work differently depending on the [settings](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/operations\/settings\/settings\/) `distributed_product_mode` setting.\n\nWhen using the regular IN, the query is sent to remote servers, and each of them runs the subqueries in the `IN` or `JOIN` clause.\n\nWhen using `GLOBAL IN` \/ `GLOBAL JOINs`, first all the subqueries are run for `GLOBAL IN` \/ `GLOBAL JOINs`, and the results are collected in temporary tables. Then the temporary tables are sent to each remote server, where the queries are run using this temporary data.\n\nFor a non-distributed query, use the regular `IN` \/ `JOIN`.\n\nBe careful when using subqueries in the `IN` \/ `JOIN` clauses for distributed query processing.\n\nLet's look at some examples. Assume that each server in the cluster has a normal **local\\_table**. Each server also has a **distributed\\_table** table with the **Distributed** type, which looks at all the servers in the cluster.\n\nFor a query to the **distributed\\_table**, the query will be sent to all the remote servers and run on them using the **local\\_table**.\n\nFor example, the query\n```\nSELECT uniq(UserID) FROM distributed_table\n```\nwill be sent to all remote servers as\n```\nSELECT uniq(UserID) FROM local_table\n```\nand run on each of them in parallel, until it reaches the stage where intermediate results can be combined. Then the intermediate results will be returned to the requestor server and merged on it, and the final result will be sent to the client.\n\nNow let's examine a query with IN:\n```\nSELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34)\n```\n- Calculation of the intersection of audiences of two sites.\n\nThis query will be sent to all remote servers as\n```\nSELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34)\n```\nIn other words, the data set in the IN clause will be collected on each server independently, only across the data that is stored locally on each of the servers.\n\nThis will work correctly and optimally if you are prepared for this case and have spread data across the cluster servers such that the data for a single UserID resides entirely on a single server. In this case, all the necessary data will be available locally on each server. Otherwise, the result will be inaccurate. We refer to this variation of the query as \"local IN\".\n\nTo correct how the query works when data is spread randomly across the cluster servers, you could specify **distributed\\_table** inside a subquery. The query would look like this:\n```\nSELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34)\n```\nThis query will be sent to all remote servers as\n```\nSELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34)\n```\nThe subquery will begin running on each remote server. Since the subquery uses a distributed table, the subquery that is on each remote server will be resent to every remote server as\n```\nSELECT UserID FROM local_table WHERE CounterID = 34\n```\nFor example, if you have a cluster of 100 servers, executing the entire query will require 10,000 elementary requests, which is generally considered unacceptable.\n\nIn such cases, you should always use GLOBAL IN instead of IN. Let's look at how it works for the query\n```\nSELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID GLOBAL IN (SELECT UserID FROM distributed_table WHERE CounterID = 34)\n```\nThe requestor server will run the subquery\n```\nSELECT UserID FROM distributed_table WHERE CounterID = 34\n```\nand the result will be put in a temporary table in RAM. Then the request will be sent to each remote server as\n```\nSELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID GLOBAL IN _data1\n```\nand the temporary table `_data1` will be sent to every remote server with the query (the name of the temporary table is implementation-defined).\n\nThis is more optimal than using the normal IN. However, keep the following points in mind:\n\n1. When creating a temporary table, data is not made unique. To reduce the volume of data transmitted over the network, specify DISTINCT in the subquery. (You don't need to do this for a normal IN.)\n2. The temporary table will be sent to all the remote servers. Transmission does not account for network topology. For example, if 10 remote servers reside in a datacenter that is very remote in relation to the requestor server, the data will be sent 10 times over the channel to the remote datacenter. Try to avoid large data sets when using GLOBAL IN.\n3. When transmitting data to remote servers, restrictions on network bandwidth are not configurable. You might overload the network.\n4. Try to distribute data across servers so that you don't need to use GLOBAL IN on a regular basis.\n5. If you need to use GLOBAL IN often, plan the location of the ClickHouse cluster so that a single group of replicas resides in no more than one data center with a fast network between them, so that a query can be processed entirely within a single data center.\n\nIt also makes sense to specify a local table in the `GLOBAL IN` clause, in case this local table is only available on the requestor server and you want to use data from it on remote servers.\n\n### Extreme Values[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#extreme-values \"Permanent link\")\nIn addition to results, you can also get minimum and maximum values for the results columns. To do this, set the **extremes** setting to 1. Minimums and maximums are calculated for numeric types, dates, and dates with times. For other columns, the default values are output.\n\nAn extra two rows are calculated – the minimums and maximums, respectively. These extra two rows are output in JSON\\*, TabSeparated\\*, and Pretty\\* formats, separate from the other rows. They are not output for other formats.\n\nIn JSON\\* formats, the extreme values are output in a separate 'extremes' field. In TabSeparated\\* formats, the row comes after the main result, and after 'totals' if present. It is preceded by an empty row (after the other data). In Pretty\\* formats, the row is output as a separate table after the main result, and after 'totals' if present.\n\nExtreme values are calculated for rows that have passed through LIMIT. However, when using 'LIMIT offset, size', the rows before 'offset' are included in 'extremes'. In stream requests, the result may also include a small number of rows that passed through LIMIT.\n\n### Notes[¶](http:\/\/devdoc.net\/database\/ClickhouseDocs_19.4.1.3-docs\/query_language\/select\/#notes \"Permanent link\")\nThe `GROUP BY` and `ORDER BY` clauses do not support positional arguments. This contradicts MySQL, but conforms to standard SQL. For example, `GROUP BY 1, 2` will be interpreted as grouping by constants (i.e. aggregation of all rows into one).\n\nYou can use synonyms (`AS` aliases) in any part of a query.\n\nYou can put an asterisk in any part of a query instead of an expression. When the query is analyzed, the asterisk is expanded to a list of all table columns (excluding the `MATERIALIZED` and `ALIAS` columns). There are only a few cases when using an asterisk is justified:\n\n- When creating a table dump.\n- For tables containing just a few columns, such as system tables.\n- For getting information about what columns are in a table. In this case, set `LIMIT 1`. But it is better to use the `DESC TABLE` query.\n- When there is strong filtration on a small number of columns using `PREWHERE`.\n- In subqueries (since columns that aren't needed for the external query are excluded from subqueries).\n\nIn all other cases, we don't recommend using the asterisk, since it only gives you the drawbacks of a columnar DBMS instead of the advantages. In other words using the asterisk is not recommended.\n\n[Original article](https:\/\/clickhouse.yandex\/docs\/en\/query_language\/select\/)","meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

22 days ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	0.7 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/
Last Crawled	2026-03-21 05:50:52 (22 days ago)
First Indexed	2020-05-22 09:58:33 (5 years ago)
HTTP Status Code	200
Meta Title	SELECT - ClickHouse Documentation
Meta Description	null
Meta Canonical	null
Boilerpipe Text	SELECT Queries Syntax ¶ SELECT performs data retrieval. SELECT [ DISTINCT ] expr_list [ FROM [ db .] table \| ( subquery ) \| table_function ] [ FINAL ] [ SAMPLE sample_coeff ] [ ARRAY JOIN ...] [ GLOBAL ] [ ANY \| ALL ] [ INNER \| LEFT \| RIGHT \| FULL \| CROSS ] [ OUTER ] JOIN ( subquery ) \| table USING columns_list [ PREWHERE expr ] [ WHERE expr ] [ GROUP BY expr_list ] [ WITH TOTALS ] [ HAVING expr ] [ ORDER BY expr_list ] [ LIMIT [ n , ] m ] [ UNION ALL ...] [ INTO OUTFILE filename ] [ FORMAT format ] [ LIMIT n BY columns ] All the clauses are optional, except for the required list of expressions immediately after SELECT. The clauses below are described in almost the same order as in the query execution conveyor. If the query omits the DISTINCT , GROUP BY and ORDER BY clauses and the IN and JOIN subqueries, the query will be completely stream processed, using O(1) amount of RAM. Otherwise, the query might consume a lot of RAM if the appropriate restrictions are not specified: max_memory_usage , max_rows_to_group_by , max_rows_to_sort , max_rows_in_distinct , max_bytes_in_distinct , max_rows_in_set , max_bytes_in_set , max_rows_in_join , max_bytes_in_join , max_bytes_before_external_sort , max_bytes_before_external_group_by . For more information, see the section "Settings". It is possible to use external sorting (saving temporary tables to a disk) and external aggregation. The system does not have "merge join" . FROM Clause ¶ If the FROM clause is omitted, data will be read from the system.one table. The 'system.one' table contains exactly one row (this table fulfills the same purpose as the DUAL table found in other DBMSs). The FROM clause specifies the table to read data from, or a subquery, or a table function; ARRAY JOIN and the regular JOIN may also be included (see below). Instead of a table, the SELECT subquery may be specified in brackets. In this case, the subquery processing pipeline will be built into the processing pipeline of an external query. In contrast to standard SQL, a synonym does not need to be specified after a subquery. For compatibility, it is possible to write 'AS name' after a subquery, but the specified name isn't used anywhere. A table function may be specified instead of a table. For more information, see the section "Table functions". To execute a query, all the columns listed in the query are extracted from the appropriate table. Any columns not needed for the external query are thrown out of the subqueries. If a query does not list any columns (for example, SELECT count() FROM t), some column is extracted from the table anyway (the smallest one is preferred), in order to calculate the number of rows. The FINAL modifier can be used only for a SELECT from a CollapsingMergeTree table. When you specify FINAL, data is selected fully "collapsed". Keep in mind that using FINAL leads to a selection that includes columns related to the primary key, in addition to the columns specified in the SELECT. Additionally, the query will be executed in a single stream, and data will be merged during query execution. This means that when using FINAL, the query is processed more slowly. In most cases, you should avoid using FINAL. For more information, see the section "CollapsingMergeTree engine". SAMPLE Clause ¶ The SAMPLE clause allows for approximated query processing. Approximated query processing is only supported by the tables in the MergeTree family, and only if the sampling expression was specified during table creation (see MergeTree engine ). The features of data sampling are listed below: Data sampling is a deterministic mechanism. The result of the same SELECT .. SAMPLE query is always the same. Sampling works consistently for different tables. For tables with a single sampling key, a sample with the same coefficient always selects the same subset of possible data. For example, a sample of user IDs takes rows with the same subset of all the possible user IDs from different tables. This allows using the sample in subqueries in the IN clause, as well as for manually correlating results of different queries with samples. Sampling allows reading less data from a disk. Note that for this you must specify the sampling key correctly. For more details see Creating a MergeTree Table . The SAMPLE clause can be specified in several ways: SAMPLE k , where k is a decimal number from 0 to 1. The query is executed on k percent of data. For example, SAMPLE 0.1 runs the query on 10% of data. Read more SAMPLE n , where n is a sufficiently large integer. The query is executed on a sample of at least n rows (but not significantly more than this). For example, SAMPLE 10000000 runs the query on a minimum of 10,000,000 rows. Read more SAMPLE k OFFSET m where k and m are numbers from 0 to 1. The query is executed on a sample of k percent of the data. The data used for the sample is offset by m percent. Read more SAMPLE k ¶ In a SAMPLE k clause, k is a percent amount of data that the sample is taken from. The example is shown below: SELECT Title , count () * 10 AS PageViews FROM hits_distributed SAMPLE 0 . 1 WHERE CounterID = 34 GROUP BY Title ORDER BY PageViews DESC LIMIT 1000 In this example, the query is executed on a sample from 0.1 (10%) of data. Values of aggregate functions are not corrected automatically, so to get an approximate result, the value 'count()' is manually multiplied by 10. SAMPLE n ¶ In this case, the query is executed on a sample of at least n rows, where n is a sufficiently large integer. For example, SAMPLE 10000000 . Since the minimum unit for data reading is one granule (its size is set by the index_granularity setting), it makes sense to set a sample that is much larger than the size of the granule. When using the SAMPLE n clause, the relative coefficient is calculated dynamically. Since you do not know which relative percent of data was processed, you do not know the coefficient the aggregate functions should be multiplied by (for example, you do not know if the SAMPLE 1000000 was taken from a set of 10,000,000 rows or from a set of 1,000,000,000 rows). In this case, use the _sample_factor column to get the approximate result. The _sample_factor is a virtual column that ClickHouse stores relative coefficients in. This column is created automatically when you create a table with the specified sampling key. The usage example is shown below: SELECT sum ( Duration * _sample_factor ) FROM visits SAMPLE 10000000 If you need to get the approximate count of rows in a SELECT .. SAMPLE n query, get the sum() of _sample_factor column instead of counting count(column * _sample_factor) value. For example: SELECT sum ( _sample_factor ) FROM visits SAMPLE 10000000 Note that to calculate the average in a SELECT .. SAMPLE n query, you do not need to use _sample_factor column: SELECT avg ( Duration ) FROM visits SAMPLE 10000000 SAMPLE k OFFSET m ¶ You can specify the SAMPLE k OFFSET m clause, where k and m are numbers from 0 to 1. Examples are shown below. Example 1. SAMPLE 1 / 10 In this example, the sample is the 1/10th of all data: [++------------------] Example 2. SAMPLE 1 / 10 OFFSET 1 / 2 Here, the sample of 10% is taken from the second half of data. [----------++--------] ARRAY JOIN Clause ¶ Allows executing JOIN with an array or nested data structure. The intent is similar to the 'arrayJoin' function, but its functionality is broader. ARRAY JOIN is essentially INNER JOIN with an array. Example: :) CREATE TABLE arrays_test (s String, arr Array(UInt8)) ENGINE = Memory CREATE TABLE arrays_test ( s String, arr Array(UInt8) ) ENGINE = Memory Ok. 0 rows in set. Elapsed: 0.001 sec. :) INSERT INTO arrays_test VALUES ('Hello', [1,2]), ('World', [3,4,5]), ('Goodbye', []) INSERT INTO arrays_test VALUES Ok. 3 rows in set. Elapsed: 0.001 sec. :) SELECT * FROM arrays_test SELECT * FROM arrays_test ┌─s───────┬─arr─────┐ │ Hello │ [1,2] │ │ World │ [3,4,5] │ │ Goodbye │ [] │ └─────────┴─────────┘ 3 rows in set. Elapsed: 0.001 sec. :) SELECT s, arr FROM arrays_test ARRAY JOIN arr SELECT s, arr FROM arrays_test ARRAY JOIN arr ┌─s─────┬─arr─┐ │ Hello │ 1 │ │ Hello │ 2 │ │ World │ 3 │ │ World │ 4 │ │ World │ 5 │ └───────┴─────┘ 5 rows in set. Elapsed: 0.001 sec. An alias can be specified for an array in the ARRAY JOIN clause. In this case, an array item can be accessed by this alias, but the array itself by the original name. Example: :) SELECT s, arr, a FROM arrays_test ARRAY JOIN arr AS a SELECT s, arr, a FROM arrays_test ARRAY JOIN arr AS a ┌─s─────┬─arr─────┬─a─┐ │ Hello │ [1,2] │ 1 │ │ Hello │ [1,2] │ 2 │ │ World │ [3,4,5] │ 3 │ │ World │ [3,4,5] │ 4 │ │ World │ [3,4,5] │ 5 │ └───────┴─────────┴───┘ 5 rows in set. Elapsed: 0.001 sec. Multiple arrays of the same size can be comma-separated in the ARRAY JOIN clause. In this case, JOIN is performed with them simultaneously (the direct sum, not the direct product). Example: :) SELECT s, arr, a, num, mapped FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(x -> x + 1, arr) AS mapped SELECT s, arr, a, num, mapped FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(lambda(tuple(x), plus(x, 1)), arr) AS mapped ┌─s─────┬─arr─────┬─a─┬─num─┬─mapped─┐ │ Hello │ [1,2] │ 1 │ 1 │ 2 │ │ Hello │ [1,2] │ 2 │ 2 │ 3 │ │ World │ [3,4,5] │ 3 │ 1 │ 4 │ │ World │ [3,4,5] │ 4 │ 2 │ 5 │ │ World │ [3,4,5] │ 5 │ 3 │ 6 │ └───────┴─────────┴───┴─────┴────────┘ 5 rows in set. Elapsed: 0.002 sec. :) SELECT s, arr, a, num, arrayEnumerate(arr) FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num SELECT s, arr, a, num, arrayEnumerate(arr) FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num ┌─s─────┬─arr─────┬─a─┬─num─┬─arrayEnumerate(arr)─┐ │ Hello │ [1,2] │ 1 │ 1 │ [1,2] │ │ Hello │ [1,2] │ 2 │ 2 │ [1,2] │ │ World │ [3,4,5] │ 3 │ 1 │ [1,2,3] │ │ World │ [3,4,5] │ 4 │ 2 │ [1,2,3] │ │ World │ [3,4,5] │ 5 │ 3 │ [1,2,3] │ └───────┴─────────┴───┴─────┴─────────────────────┘ 5 rows in set. Elapsed: 0.002 sec. ARRAY JOIN also works with nested data structures. Example: :) CREATE TABLE nested_test (s String, nest Nested(x UInt8, y UInt32)) ENGINE = Memory CREATE TABLE nested_test ( s String, nest Nested( x UInt8, y UInt32) ) ENGINE = Memory Ok. 0 rows in set. Elapsed: 0.006 sec. :) INSERT INTO nested_test VALUES ('Hello', [1,2], [10,20]), ('World', [3,4,5], [30,40,50]), ('Goodbye', [], []) INSERT INTO nested_test VALUES Ok. 3 rows in set. Elapsed: 0.001 sec. :) SELECT * FROM nested_test SELECT * FROM nested_test ┌─s───────┬─nest.x──┬─nest.y─────┐ │ Hello │ [1,2] │ [10,20] │ │ World │ [3,4,5] │ [30,40,50] │ │ Goodbye │ [] │ [] │ └─────────┴─────────┴────────────┘ 3 rows in set. Elapsed: 0.001 sec. :) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest SELECT s, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN nest ┌─s─────┬─nest.x─┬─nest.y─┐ │ Hello │ 1 │ 10 │ │ Hello │ 2 │ 20 │ │ World │ 3 │ 30 │ │ World │ 4 │ 40 │ │ World │ 5 │ 50 │ └───────┴────────┴────────┘ 5 rows in set. Elapsed: 0.001 sec. When specifying names of nested data structures in ARRAY JOIN, the meaning is the same as ARRAY JOIN with all the array elements that it consists of. Example: :) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x, nest.y SELECT s, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN `nest.x`, `nest.y` ┌─s─────┬─nest.x─┬─nest.y─┐ │ Hello │ 1 │ 10 │ │ Hello │ 2 │ 20 │ │ World │ 3 │ 30 │ │ World │ 4 │ 40 │ │ World │ 5 │ 50 │ └───────┴────────┴────────┘ 5 rows in set. Elapsed: 0.001 sec. This variation also makes sense: :) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x SELECT s, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN `nest.x` ┌─s─────┬─nest.x─┬─nest.y─────┐ │ Hello │ 1 │ [10,20] │ │ Hello │ 2 │ [10,20] │ │ World │ 3 │ [30,40,50] │ │ World │ 4 │ [30,40,50] │ │ World │ 5 │ [30,40,50] │ └───────┴────────┴────────────┘ 5 rows in set. Elapsed: 0.001 sec. An alias may be used for a nested data structure, in order to select either the JOIN result or the source array. Example: :) SELECT s, n.x, n.y, nest.x, nest.y FROM nested_test ARRAY JOIN nest AS n SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN nest AS n ┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┐ │ Hello │ 1 │ 10 │ [1,2] │ [10,20] │ │ Hello │ 2 │ 20 │ [1,2] │ [10,20] │ │ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │ │ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │ │ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │ └───────┴─────┴─────┴─────────┴────────────┘ 5 rows in set. Elapsed: 0.001 sec. Example of using the arrayEnumerate function: :) SELECT s, n.x, n.y, nest.x, nest.y, num FROM nested_test ARRAY JOIN nest AS n, arrayEnumerate(nest.x) AS num SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`, num FROM nested_test ARRAY JOIN nest AS n, arrayEnumerate(`nest.x`) AS num ┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┬─num─┐ │ Hello │ 1 │ 10 │ [1,2] │ [10,20] │ 1 │ │ Hello │ 2 │ 20 │ [1,2] │ [10,20] │ 2 │ │ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │ 1 │ │ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │ 2 │ │ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │ 3 │ └───────┴─────┴─────┴─────────┴────────────┴─────┘ 5 rows in set. Elapsed: 0.002 sec. The query can only specify a single ARRAY JOIN clause. The corresponding conversion can be performed before the WHERE/PREWHERE clause (if its result is needed in this clause), or after completing WHERE/PREWHERE (to reduce the volume of calculations). JOIN Clause ¶ Joins the data in the normal SQL JOIN sense. Note Not related to ARRAY JOIN . SELECT < expr_list > FROM < left_subquery > [ GLOBAL ] [ ANY \| ALL ] [ INNER \| LEFT \| RIGHT \| FULL \| CROSS ] [ OUTER ] JOIN < right_subquery > ( ON < expr_list > ) \| ( USING < column_list > ) ... The table names can be specified instead of <left_subquery> and <right_subquery> . This is equivalent to the SELECT * FROM table subquery, except in a special case when the table has the Join engine – an array prepared for joining. Supported types of JOIN INNER JOIN (or JOIN ) LEFT JOIN (or LEFT OUTER JOIN ) RIGHT JOIN (or RIGHT OUTER JOIN ) FULL JOIN (or FULL OUTER JOIN ) CROSS JOIN (or , ) See standard SQL JOIN description. ANY or ALL strictness If ALL is specified and the right table has several matching rows, the data will be multiplied by the number of these rows. This is the normal JOIN behavior for standard SQL. If ANY is specified and the right table has several matching rows, only the first one found is joined. If the right table has only one matching row, the results of ANY and ALL are the same. To set the default strictness value, use the session configuration parameter join_default_strictness . GLOBAL JOIN When using a normal JOIN , the query is sent to remote servers. Subqueries are run on each of them in order to make the right table, and the join is performed with this table. In other words, the right table is formed on each server separately. When using GLOBAL ... JOIN , first the requestor server runs a subquery to calculate the right table. This temporary table is passed to each remote server, and queries are run on them using the temporary data that was transmitted. Be careful when using GLOBAL . For more information, see the section Distributed subqueries . Usage Recommendations All columns that are not needed for the JOIN are deleted from the subquery. When running a JOIN , there is no optimization of the order of execution in relation to other stages of the query. The join (a search in the right table) is run before filtering in WHERE and before aggregation. In order to explicitly set the processing order, we recommend running a JOIN subquery with a subquery. Example: SELECT CounterID , hits , visits FROM ( SELECT CounterID , count () AS hits FROM test . hits GROUP BY CounterID ) ANY LEFT JOIN ( SELECT CounterID , sum ( Sign ) AS visits FROM test . visits GROUP BY CounterID ) USING CounterID ORDER BY hits DESC LIMIT 10 ┌─CounterID─┬───hits─┬─visits─┐ │ 1143050 │ 523264 │ 13665 │ │ 731962 │ 475698 │ 102716 │ │ 722545 │ 337212 │ 108187 │ │ 722889 │ 252197 │ 10547 │ │ 2237260 │ 196036 │ 9522 │ │ 23057320 │ 147211 │ 7689 │ │ 722818 │ 90109 │ 17847 │ │ 48221 │ 85379 │ 4652 │ │ 19762435 │ 77807 │ 7026 │ │ 722884 │ 77492 │ 11056 │ └───────────┴────────┴────────┘ Subqueries don't allow you to set names or use them for referencing a column from a specific subquery. The columns specified in USING must have the same names in both subqueries, and the other columns must be named differently. You can use aliases to change the names of columns in subqueries (the example uses the aliases 'hits' and 'visits'). The USING clause specifies one or more columns to join, which establishes the equality of these columns. The list of columns is set without brackets. More complex join conditions are not supported. The right table (the subquery result) resides in RAM. If there isn't enough memory, you can't run a JOIN . Each time a query is run with the same JOIN , the subquery is run again because the result is not cached. To avoid this, use the special Join table engine, which is a prepared array for joining that is always in RAM. In some cases, it is more efficient to use IN instead of JOIN . Among the various types of JOIN , the most efficient is ANY LEFT JOIN , then ANY INNER JOIN . The least efficient are ALL LEFT JOIN and ALL INNER JOIN . If you need a JOIN for joining with dimension tables (these are relatively small tables that contain dimension properties, such as names for advertising campaigns), a JOIN might not be very convenient due to the bulky syntax and the fact that the right table is re-accessed for every query. For such cases, there is an "external dictionaries" feature that you should use instead of JOIN . For more information, see the section External dictionaries . Processing of Empty or NULL Cells ¶ While joining tables, the empty cells may appear. The setting join_use_nulls define how ClickHouse fills these cells. If the JOIN keys are Nullable fields, the rows where at least one of the keys has the value NULL are not joined. WHERE Clause ¶ If there is a WHERE clause, it must contain an expression with the UInt8 type. This is usually an expression with comparison and logical operators. This expression will be used for filtering data before all other transformations. If indexes are supported by the database table engine, the expression is evaluated on the ability to use indexes. PREWHERE Clause ¶ This clause has the same meaning as the WHERE clause. The difference is in which data is read from the table. When using PREWHERE, first only the columns necessary for executing PREWHERE are read. Then the other columns are read that are needed for running the query, but only those blocks where the PREWHERE expression is true. It makes sense to use PREWHERE if there are filtration conditions that are used by a minority of the columns in the query, but that provide strong data filtration. This reduces the volume of data to read. For example, it is useful to write PREWHERE for queries that extract a large number of columns, but that only have filtration for a few columns. PREWHERE is only supported by tables from the MergeTree family. A query may simultaneously specify PREWHERE and WHERE. In this case, PREWHERE precedes WHERE. If the 'optimize_move_to_prewhere' setting is set to 1 and PREWHERE is omitted, the system uses heuristics to automatically move parts of expressions from WHERE to PREWHERE. GROUP BY Clause ¶ This is one of the most important parts of a column-oriented DBMS. If there is a GROUP BY clause, it must contain a list of expressions. Each expression will be referred to here as a "key". All the expressions in the SELECT, HAVING, and ORDER BY clauses must be calculated from keys or from aggregate functions. In other words, each column selected from the table must be used either in keys or inside aggregate functions. If a query contains only table columns inside aggregate functions, the GROUP BY clause can be omitted, and aggregation by an empty set of keys is assumed. Example: SELECT count (), median ( FetchTiming > 60 ? 60 : FetchTiming ), count () - sum ( Refresh ) FROM hits However, in contrast to standard SQL, if the table doesn't have any rows (either there aren't any at all, or there aren't any after using WHERE to filter), an empty result is returned, and not the result from one of the rows containing the initial values of aggregate functions. As opposed to MySQL (and conforming to standard SQL), you can't get some value of some column that is not in a key or aggregate function (except constant expressions). To work around this, you can use the 'any' aggregate function (get the first encountered value) or 'min/max'. Example: SELECT domainWithoutWWW ( URL ) AS domain , count (), any ( Title ) AS title -- getting the first occurred page header for each domain. FROM hits GROUP BY domain For every different key value encountered, GROUP BY calculates a set of aggregate function values. GROUP BY is not supported for array columns. A constant can't be specified as arguments for aggregate functions. Example: sum(1). Instead of this, you can get rid of the constant. Example: count() . NULL processing ¶ For grouping, ClickHouse interprets NULL as a value, and NULL=NULL . Here's an example to show what this means. Assume you have this table: ┌─x─┬────y─┐ │ 1 │ 2 │ │ 2 │ ᴺᵁᴸᴸ │ │ 3 │ 2 │ │ 3 │ 3 │ │ 3 │ ᴺᵁᴸᴸ │ └───┴──────┘ The query SELECT sum(x), y FROM t_null_big GROUP BY y results in: ┌─sum(x)─┬────y─┐ │ 4 │ 2 │ │ 3 │ 3 │ │ 5 │ ᴺᵁᴸᴸ │ └────────┴──────┘ You can see that GROUP BY for У = NULL summed up x , as if NULL is this value. If you pass several keys to GROUP BY , the result will give you all the combinations of the selection, as if NULL were a specific value. WITH TOTALS Modifier ¶ If the WITH TOTALS modifier is specified, another row will be calculated. This row will have key columns containing default values (zeros or empty lines), and columns of aggregate functions with the values calculated across all the rows (the "total" values). This extra row is output in JSON, TabSeparated, and Pretty formats, separately from the other rows. In the other formats, this row is not output. In JSON* formats, this row is output as a separate 'totals' field. In TabSeparated* formats, the row comes after the main result, preceded by an empty row (after the other data). In Pretty* formats, the row is output as a separate table after the main result. WITH TOTALS can be run in different ways when HAVING is present. The behavior depends on the 'totals_mode' setting. By default, totals_mode = 'before_having' . In this case, 'totals' is calculated across all rows, including the ones that don't pass through HAVING and 'max_rows_to_group_by'. The other alternatives include only the rows that pass through HAVING in 'totals', and behave differently with the setting max_rows_to_group_by and group_by_overflow_mode = 'any' . after_having_exclusive – Don't include rows that didn't pass through max_rows_to_group_by . In other words, 'totals' will have less than or the same number of rows as it would if max_rows_to_group_by were omitted. after_having_inclusive – Include all the rows that didn't pass through 'max_rows_to_group_by' in 'totals'. In other words, 'totals' will have more than or the same number of rows as it would if max_rows_to_group_by were omitted. after_having_auto – Count the number of rows that passed through HAVING. If it is more than a certain amount (by default, 50%), include all the rows that didn't pass through 'max_rows_to_group_by' in 'totals'. Otherwise, do not include them. totals_auto_threshold – By default, 0.5. The coefficient for after_having_auto . If max_rows_to_group_by and group_by_overflow_mode = 'any' are not used, all variations of after_having are the same, and you can use any of them (for example, after_having_auto ). You can use WITH TOTALS in subqueries, including subqueries in the JOIN clause (in this case, the respective total values are combined). GROUP BY in External Memory ¶ You can enable dumping temporary data to the disk to restrict memory usage during GROUP BY. The max_bytes_before_external_group_by setting determines the threshold RAM consumption for dumping GROUP BY temporary data to the file system. If set to 0 (the default), it is disabled. When using max_bytes_before_external_group_by , we recommend that you set max_memory_usage about twice as high. This is necessary because there are two stages to aggregation: reading the date and forming intermediate data (1) and merging the intermediate data (2). Dumping data to the file system can only occur during stage 1. If the temporary data wasn't dumped, then stage 2 might require up to the same amount of memory as in stage 1. For example, if max_memory_usage was set to 10000000000 and you want to use external aggregation, it makes sense to set max_bytes_before_external_group_by to 10000000000, and max_memory_usage to 20000000000. When external aggregation is triggered (if there was at least one dump of temporary data), maximum consumption of RAM is only slightly more than max_bytes_before_external_group_by . With distributed query processing, external aggregation is performed on remote servers. In order for the requestor server to use only a small amount of RAM, set distributed_aggregation_memory_efficient to 1. When merging data flushed to the disk, as well as when merging results from remote servers when the distributed_aggregation_memory_efficient setting is enabled, consumes up to 1/256 * the number of threads from the total amount of RAM. When external aggregation is enabled, if there was less than max_bytes_before_external_group_by of data (i.e. data was not flushed), the query runs just as fast as without external aggregation. If any temporary data was flushed, the run time will be several times longer (approximately three times). If you have an ORDER BY with a small LIMIT after GROUP BY, then the ORDER BY CLAUSE will not use significant amounts of RAM. But if the ORDER BY doesn't have LIMIT, don't forget to enable external sorting ( max_bytes_before_external_sort ). LIMIT N BY Clause ¶ LIMIT N BY COLUMNS selects the top N rows for each group of COLUMNS. LIMIT N BY is not related to LIMIT; they can both be used in the same query. The key for LIMIT N BY can contain any number of columns or expressions. Example: SELECT domainWithoutWWW ( URL ) AS domain , domainWithoutWWW ( REFERRER_URL ) AS referrer , device_type , count () cnt FROM hits GROUP BY domain , referrer , device_type ORDER BY cnt DESC LIMIT 5 BY domain , device_type LIMIT 100 The query will select the top 5 referrers for each domain, device_type pair, but not more than 100 rows ( LIMIT n BY + LIMIT ). HAVING Clause ¶ Allows filtering the result received after GROUP BY, similar to the WHERE clause. WHERE and HAVING differ in that WHERE is performed before aggregation (GROUP BY), while HAVING is performed after it. If aggregation is not performed, HAVING can't be used. ORDER BY Clause ¶ The ORDER BY clause contains a list of expressions, which can each be assigned DESC or ASC (the sorting direction). If the direction is not specified, ASC is assumed. ASC is sorted in ascending order, and DESC in descending order. The sorting direction applies to a single expression, not to the entire list. Example: ORDER BY Visits DESC, SearchPhrase For sorting by String values, you can specify collation (comparison). Example: ORDER BY SearchPhrase COLLATE 'tr' - for sorting by keyword in ascending order, using the Turkish alphabet, case insensitive, assuming that strings are UTF-8 encoded. COLLATE can be specified or not for each expression in ORDER BY independently. If ASC or DESC is specified, COLLATE is specified after it. When using COLLATE, sorting is always case-insensitive. We only recommend using COLLATE for final sorting of a small number of rows, since sorting with COLLATE is less efficient than normal sorting by bytes. Rows that have identical values for the list of sorting expressions are output in an arbitrary order, which can also be nondeterministic (different each time). If the ORDER BY clause is omitted, the order of the rows is also undefined, and may be nondeterministic as well. NaN and NULL sorting order: With the modifier NULLS FIRST — First NULL , then NaN , then other values. With the modifier NULLS LAST — First the values, then NaN , then NULL . Default — The same as with the NULLS LAST modifier. Example: For the table ┌─x─┬────y─┐ │ 1 │ ᴺᵁᴸᴸ │ │ 2 │ 2 │ │ 1 │ nan │ │ 2 │ 2 │ │ 3 │ 4 │ │ 5 │ 6 │ │ 6 │ nan │ │ 7 │ ᴺᵁᴸᴸ │ │ 6 │ 7 │ │ 8 │ 9 │ └───┴──────┘ Run the query SELECT * FROM t_null_nan ORDER BY y NULLS FIRST to get: ┌─x─┬────y─┐ │ 1 │ ᴺᵁᴸᴸ │ │ 7 │ ᴺᵁᴸᴸ │ │ 1 │ nan │ │ 6 │ nan │ │ 2 │ 2 │ │ 2 │ 2 │ │ 3 │ 4 │ │ 5 │ 6 │ │ 6 │ 7 │ │ 8 │ 9 │ └───┴──────┘ When floating point numbers are sorted, NaNs are separate from the other values. Regardless of the sorting order, NaNs come at the end. In other words, for ascending sorting they are placed as if they are larger than all the other numbers, while for descending sorting they are placed as if they are smaller than the rest. Less RAM is used if a small enough LIMIT is specified in addition to ORDER BY. Otherwise, the amount of memory spent is proportional to the volume of data for sorting. For distributed query processing, if GROUP BY is omitted, sorting is partially done on remote servers, and the results are merged on the requestor server. This means that for distributed sorting, the volume of data to sort can be greater than the amount of memory on a single server. If there is not enough RAM, it is possible to perform sorting in external memory (creating temporary files on a disk). Use the setting max_bytes_before_external_sort for this purpose. If it is set to 0 (the default), external sorting is disabled. If it is enabled, when the volume of data to sort reaches the specified number of bytes, the collected data is sorted and dumped into a temporary file. After all data is read, all the sorted files are merged and the results are output. Files are written to the /var/lib/clickhouse/tmp/ directory in the config (by default, but you can use the 'tmp_path' parameter to change this setting). Running a query may use more memory than 'max_bytes_before_external_sort'. For this reason, this setting must have a value significantly smaller than 'max_memory_usage'. As an example, if your server has 128 GB of RAM and you need to run a single query, set 'max_memory_usage' to 100 GB, and 'max_bytes_before_external_sort' to 80 GB. External sorting works much less effectively than sorting in RAM. SELECT Clause ¶ The expressions specified in the SELECT clause are analyzed after the calculations for all the clauses listed above are completed. More specifically, expressions are analyzed that are above the aggregate functions, if there are any aggregate functions. The aggregate functions and everything below them are calculated during aggregation (GROUP BY). These expressions work as if they are applied to separate rows in the result. DISTINCT Clause ¶ If DISTINCT is specified, only a single row will remain out of all the sets of fully matching rows in the result. The result will be the same as if GROUP BY were specified across all the fields specified in SELECT without aggregate functions. But there are several differences from GROUP BY: DISTINCT can be applied together with GROUP BY. When ORDER BY is omitted and LIMIT is defined, the query stops running immediately after the required number of different rows has been read. Data blocks are output as they are processed, without waiting for the entire query to finish running. DISTINCT is not supported if SELECT has at least one array column. DISTINCT works with NULL as if NULL were a specific value, and NULL=NULL . In other words, in the DISTINCT results, different combinations with NULL only occur once. LIMIT Clause ¶ LIMIT m allows you to select the first m rows from the result. LIMIT n , m allows you to select the first m rows from the result after skipping the first n rows. n and m must be non-negative integers. If there isn't an ORDER BY clause that explicitly sorts results, the result may be arbitrary and nondeterministic. UNION ALL Clause ¶ You can use UNION ALL to combine any number of queries. Example: SELECT CounterID , 1 AS table , toInt64 ( count ()) AS c FROM test . hits GROUP BY CounterID UNION ALL SELECT CounterID , 2 AS table , sum ( Sign ) AS c FROM test . visits GROUP BY CounterID HAVING c > 0 Only UNION ALL is supported. The regular UNION (UNION DISTINCT) is not supported. If you need UNION DISTINCT, you can write SELECT DISTINCT from a subquery containing UNION ALL. Queries that are parts of UNION ALL can be run simultaneously, and their results can be mixed together. The structure of results (the number and type of columns) must match for the queries. But the column names can differ. In this case, the column names for the final result will be taken from the first query. Type casting is performed for unions. For example, if two queries being combined have the same field with non- Nullable and Nullable types from a compatible type, the resulting UNION ALL has a Nullable type field. Queries that are parts of UNION ALL can't be enclosed in brackets. ORDER BY and LIMIT are applied to separate queries, not to the final result. If you need to apply a conversion to the final result, you can put all the queries with UNION ALL in a subquery in the FROM clause. INTO OUTFILE Clause ¶ Add the INTO OUTFILE filename clause (where filename is a string literal) to redirect query output to the specified file. In contrast to MySQL, the file is created on the client side. The query will fail if a file with the same filename already exists. This functionality is available in the command-line client and clickhouse-local (a query sent via HTTP interface will fail). The default output format is TabSeparated (the same as in the command-line client batch mode). FORMAT Clause ¶ Specify 'FORMAT format' to get data in any specified format. You can use this for convenience, or for creating dumps. For more information, see the section "Formats". If the FORMAT clause is omitted, the default format is used, which depends on both the settings and the interface used for accessing the DB. For the HTTP interface and the command-line client in batch mode, the default format is TabSeparated. For the command-line client in interactive mode, the default format is PrettyCompact (it has attractive and compact tables). When using the command-line client, data is passed to the client in an internal efficient format. The client independently interprets the FORMAT clause of the query and formats the data itself (thus relieving the network and the server from the load). IN Operators ¶ The IN , NOT IN , GLOBAL IN , and GLOBAL NOT IN operators are covered separately, since their functionality is quite rich. The left side of the operator is either a single column or a tuple. Examples: SELECT UserID IN ( 123 , 456 ) FROM ... SELECT ( CounterID , UserID ) IN (( 34 , 123 ), ( 101500 , 456 )) FROM ... If the left side is a single column that is in the index, and the right side is a set of constants, the system uses the index for processing the query. Don't list too many values explicitly (i.e. millions). If a data set is large, put it in a temporary table (for example, see the section "External data for query processing"), then use a subquery. The right side of the operator can be a set of constant expressions, a set of tuples with constant expressions (shown in the examples above), or the name of a database table or SELECT subquery in brackets. If the right side of the operator is the name of a table (for example, UserID IN users ), this is equivalent to the subquery UserID IN (SELECT * FROM users) . Use this when working with external data that is sent along with the query. For example, the query can be sent together with a set of user IDs loaded to the 'users' temporary table, which should be filtered. If the right side of the operator is a table name that has the Set engine (a prepared data set that is always in RAM), the data set will not be created over again for each query. The subquery may specify more than one column for filtering tuples. Example: SELECT ( CounterID , UserID ) IN ( SELECT CounterID , UserID FROM ...) FROM ... The columns to the left and right of the IN operator should have the same type. The IN operator and subquery may occur in any part of the query, including in aggregate functions and lambda functions. Example: SELECT EventDate , avg ( UserID IN ( SELECT UserID FROM test . hits WHERE EventDate = toDate ( '2014-03-17' ) )) AS ratio FROM test . hits GROUP BY EventDate ORDER BY EventDate ASC ┌──EventDate─┬────ratio─┐ │ 2014-03-17 │ 1 │ │ 2014-03-18 │ 0.807696 │ │ 2014-03-19 │ 0.755406 │ │ 2014-03-20 │ 0.723218 │ │ 2014-03-21 │ 0.697021 │ │ 2014-03-22 │ 0.647851 │ │ 2014-03-23 │ 0.648416 │ └────────────┴──────────┘ For each day after March 17th, count the percentage of pageviews made by users who visited the site on March 17th. A subquery in the IN clause is always run just one time on a single server. There are no dependent subqueries. NULL processing ¶ During request processing, the IN operator assumes that the result of an operation with NULL is always equal to 0 , regardless of whether NULL is on the right or left side of the operator. NULL values are not included in any dataset, do not correspond to each other and cannot be compared. Here is an example with the t_null table: ┌─x─┬────y─┐ │ 1 │ ᴺᵁᴸᴸ │ │ 2 │ 3 │ └───┴──────┘ Running the query SELECT x FROM t_null WHERE y IN (NULL,3) gives you the following result: ┌─x─┐ │ 2 │ └───┘ You can see that the row in which y = NULL is thrown out of the query results. This is because ClickHouse can't decide whether NULL is included in the (NULL,3) set, returns 0 as the result of the operation, and SELECT excludes this row from the final output. SELECT y IN (NULL, 3) FROM t_null ┌─in(y, tuple(NULL, 3))─┐ │ 0 │ │ 1 │ └───────────────────────┘ Distributed Subqueries ¶ There are two options for IN-s with subqueries (similar to JOINs): normal IN / JOIN and GLOBAL IN / GLOBAL JOIN . They differ in how they are run for distributed query processing. Attention Remember that the algorithms described below may work differently depending on the settings distributed_product_mode setting. When using the regular IN, the query is sent to remote servers, and each of them runs the subqueries in the IN or JOIN clause. When using GLOBAL IN / GLOBAL JOINs , first all the subqueries are run for GLOBAL IN / GLOBAL JOINs , and the results are collected in temporary tables. Then the temporary tables are sent to each remote server, where the queries are run using this temporary data. For a non-distributed query, use the regular IN / JOIN . Be careful when using subqueries in the IN / JOIN clauses for distributed query processing. Let's look at some examples. Assume that each server in the cluster has a normal local_table . Each server also has a distributed_table table with the Distributed type, which looks at all the servers in the cluster. For a query to the distributed_table , the query will be sent to all the remote servers and run on them using the local_table . For example, the query SELECT uniq ( UserID ) FROM distributed_table will be sent to all remote servers as SELECT uniq ( UserID ) FROM local_table and run on each of them in parallel, until it reaches the stage where intermediate results can be combined. Then the intermediate results will be returned to the requestor server and merged on it, and the final result will be sent to the client. Now let's examine a query with IN: SELECT uniq ( UserID ) FROM distributed_table WHERE CounterID = 101500 AND UserID IN ( SELECT UserID FROM local_table WHERE CounterID = 34 ) Calculation of the intersection of audiences of two sites. This query will be sent to all remote servers as SELECT uniq ( UserID ) FROM local_table WHERE CounterID = 101500 AND UserID IN ( SELECT UserID FROM local_table WHERE CounterID = 34 ) In other words, the data set in the IN clause will be collected on each server independently, only across the data that is stored locally on each of the servers. This will work correctly and optimally if you are prepared for this case and have spread data across the cluster servers such that the data for a single UserID resides entirely on a single server. In this case, all the necessary data will be available locally on each server. Otherwise, the result will be inaccurate. We refer to this variation of the query as "local IN". To correct how the query works when data is spread randomly across the cluster servers, you could specify distributed_table inside a subquery. The query would look like this: SELECT uniq ( UserID ) FROM distributed_table WHERE CounterID = 101500 AND UserID IN ( SELECT UserID FROM distributed_table WHERE CounterID = 34 ) This query will be sent to all remote servers as SELECT uniq ( UserID ) FROM local_table WHERE CounterID = 101500 AND UserID IN ( SELECT UserID FROM distributed_table WHERE CounterID = 34 ) The subquery will begin running on each remote server. Since the subquery uses a distributed table, the subquery that is on each remote server will be resent to every remote server as SELECT UserID FROM local_table WHERE CounterID = 34 For example, if you have a cluster of 100 servers, executing the entire query will require 10,000 elementary requests, which is generally considered unacceptable. In such cases, you should always use GLOBAL IN instead of IN. Let's look at how it works for the query SELECT uniq ( UserID ) FROM distributed_table WHERE CounterID = 101500 AND UserID GLOBAL IN ( SELECT UserID FROM distributed_table WHERE CounterID = 34 ) The requestor server will run the subquery SELECT UserID FROM distributed_table WHERE CounterID = 34 and the result will be put in a temporary table in RAM. Then the request will be sent to each remote server as SELECT uniq ( UserID ) FROM local_table WHERE CounterID = 101500 AND UserID GLOBAL IN _data1 and the temporary table _data1 will be sent to every remote server with the query (the name of the temporary table is implementation-defined). This is more optimal than using the normal IN. However, keep the following points in mind: When creating a temporary table, data is not made unique. To reduce the volume of data transmitted over the network, specify DISTINCT in the subquery. (You don't need to do this for a normal IN.) The temporary table will be sent to all the remote servers. Transmission does not account for network topology. For example, if 10 remote servers reside in a datacenter that is very remote in relation to the requestor server, the data will be sent 10 times over the channel to the remote datacenter. Try to avoid large data sets when using GLOBAL IN. When transmitting data to remote servers, restrictions on network bandwidth are not configurable. You might overload the network. Try to distribute data across servers so that you don't need to use GLOBAL IN on a regular basis. If you need to use GLOBAL IN often, plan the location of the ClickHouse cluster so that a single group of replicas resides in no more than one data center with a fast network between them, so that a query can be processed entirely within a single data center. It also makes sense to specify a local table in the GLOBAL IN clause, in case this local table is only available on the requestor server and you want to use data from it on remote servers. Extreme Values ¶ In addition to results, you can also get minimum and maximum values for the results columns. To do this, set the extremes setting to 1. Minimums and maximums are calculated for numeric types, dates, and dates with times. For other columns, the default values are output. An extra two rows are calculated – the minimums and maximums, respectively. These extra two rows are output in JSON, TabSeparated, and Pretty* formats, separate from the other rows. They are not output for other formats. In JSON* formats, the extreme values are output in a separate 'extremes' field. In TabSeparated* formats, the row comes after the main result, and after 'totals' if present. It is preceded by an empty row (after the other data). In Pretty* formats, the row is output as a separate table after the main result, and after 'totals' if present. Extreme values are calculated for rows that have passed through LIMIT. However, when using 'LIMIT offset, size', the rows before 'offset' are included in 'extremes'. In stream requests, the result may also include a small number of rows that passed through LIMIT. Notes ¶ The GROUP BY and ORDER BY clauses do not support positional arguments. This contradicts MySQL, but conforms to standard SQL. For example, GROUP BY 1, 2 will be interpreted as grouping by constants (i.e. aggregation of all rows into one). You can use synonyms ( AS aliases) in any part of a query. You can put an asterisk in any part of a query instead of an expression. When the query is analyzed, the asterisk is expanded to a list of all table columns (excluding the MATERIALIZED and ALIAS columns). There are only a few cases when using an asterisk is justified: When creating a table dump. For tables containing just a few columns, such as system tables. For getting information about what columns are in a table. In this case, set LIMIT 1 . But it is better to use the DESC TABLE query. When there is strong filtration on a small number of columns using PREWHERE . In subqueries (since columns that aren't needed for the external query are excluded from subqueries). In all other cases, we don't recommend using the asterisk, since it only gives you the drawbacks of a columnar DBMS instead of the advantages. In other words using the asterisk is not recommended. Original article
Markdown	[![](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/images/logo.svg)](http://devdoc.net/ "ClickHouse Documentation") ClickHouse Documentation SELECT Type to start searching ClickHouse Documentation - Introduction Introduction - [Overview](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/ "Overview") - [Distinctive Features of ClickHouse](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/introduction/distinctive_features/ "Distinctive Features of ClickHouse") - [ClickHouse Features that Can Be Considered Disadvantages](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/introduction/features_considered_disadvantages/ "ClickHouse Features that Can Be Considered Disadvantages") - [Performance](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/introduction/performance/ "Performance") - [The Yandex.Metrica Task](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/introduction/ya_metrika_task/ "The Yandex.Metrica Task") - Getting Started Getting Started - [Deploying and Running](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/getting_started/ "Deploying and Running") - Example Datasets Example Datasets - [OnTime](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/getting_started/example_datasets/ontime/ "OnTime") - [New York Taxi Data](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/getting_started/example_datasets/nyc_taxi/ "New York Taxi Data") - [AMPLab Big Data Benchmark](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/getting_started/example_datasets/amplab_benchmark/ "AMPLab Big Data Benchmark") - [WikiStat](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/getting_started/example_datasets/wikistat/ "WikiStat") - [Terabyte Click Logs from Criteo](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/getting_started/example_datasets/criteo/ "Terabyte Click Logs from Criteo") - [Star Schema Benchmark](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/getting_started/example_datasets/star_schema/ "Star Schema Benchmark") - [Yandex.Metrica Data](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/getting_started/example_datasets/metrica/ "Yandex.Metrica Data") - Interfaces Interfaces - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/ "Introduction") - [Command-Line Client](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/cli/ "Command-Line Client") - [Native Interface (TCP)](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/tcp/ "Native Interface (TCP)") - [HTTP Interface](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/http/ "HTTP Interface") - [Input and Output Formats](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/formats/ "Input and Output Formats") - [JDBC Driver](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/jdbc/ "JDBC Driver") - [ODBC Driver](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/odbc/ "ODBC Driver") - Third-Party Third-Party - [Client Libraries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/third-party/client_libraries/ "Client Libraries") - [Integrations](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/third-party/integrations/ "Integrations") - [Visual Interfaces](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/third-party/gui/ "Visual Interfaces") - [Proxies](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/interfaces/third-party/proxy/ "Proxies") - Data Types Data Types - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/ "Introduction") - [UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/int_uint/ "UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64") - [Float32, Float64](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/float/ "Float32, Float64") - [Decimal](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/decimal/ "Decimal") - [Boolean](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/boolean/ "Boolean") - [String](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/string/ "String") - [FixedString(N)](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/fixedstring/ "FixedString(N)") - [UUID](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/uuid/ "UUID") - [Date](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/date/ "Date") - [DateTime](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/datetime/ "DateTime") - [Enum](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/enum/ "Enum") - [Array(T)](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/array/ "Array(T)") - [AggregateFunction(name, types\_of\_arguments...)](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/nested_data_structures/aggregatefunction/ "AggregateFunction(name, types_of_arguments...)") - [Tuple(T1, T2, ...)](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/tuple/ "Tuple(T1, T2, ...)") - [Nullable](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/nullable/ "Nullable") - Nested Data Structures Nested Data Structures - [Nested(Name1 Type1, Name2 Type2, ...)](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/nested_data_structures/nested/ "Nested(Name1 Type1, Name2 Type2, ...)") - Special Data Types Special Data Types - [Expression](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/special_data_types/expression/ "Expression") - [Set](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/special_data_types/set/ "Set") - [Nothing](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/special_data_types/nothing/ "Nothing") - Table Engines Table Engines - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/ "Introduction") - MergeTree Family MergeTree Family - [MergeTree](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/mergetree/ "MergeTree") - [Data Replication](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/replication/ "Data Replication") - [Custom Partitioning Key](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/custom_partitioning_key/ "Custom Partitioning Key") - [ReplacingMergeTree](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/replacingmergetree/ "ReplacingMergeTree") - [SummingMergeTree](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/summingmergetree/ "SummingMergeTree") - [AggregatingMergeTree](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/aggregatingmergetree/ "AggregatingMergeTree") - [CollapsingMergeTree](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/collapsingmergetree/ "CollapsingMergeTree") - [VersionedCollapsingMergeTree](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/versionedcollapsingmergetree/ "VersionedCollapsingMergeTree") - [GraphiteMergeTree](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/graphitemergetree/ "GraphiteMergeTree") - Log Family Log Family - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/log_family/ "Introduction") - [StripeLog](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/stripelog/ "StripeLog") - [Log](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/log/ "Log") - [TinyLog](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/tinylog/ "TinyLog") - Integrations Integrations - [Kafka](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/kafka/ "Kafka") - [MySQL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/mysql/ "MySQL") - Special Special - [Distributed](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/distributed/ "Distributed") - [External data](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/external_data/ "External data") - [Dictionary](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/dictionary/ "Dictionary") - [Merge](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/merge/ "Merge") - [File](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/file/ "File") - [Null](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/null/ "Null") - [Set](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/set/ "Set") - [Join](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/join/ "Join") - [URL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/url/ "URL") - [View](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/view/ "View") - [MaterializedView](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/materializedview/ "MaterializedView") - [Memory](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/memory/ "Memory") - [Buffer](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/buffer/ "Buffer") - SQL Reference SQL Reference - SELECT [SELECT](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/ "SELECT") Table of contents - [FROM Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#from-clause "FROM Clause") - [SAMPLE Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-clause "SAMPLE Clause") - [SAMPLE k](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-k "SAMPLE k") - [SAMPLE n](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-n "SAMPLE n") - [SAMPLE k OFFSET m](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-offset "SAMPLE k OFFSET m") - [ARRAY JOIN Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-array-join-clause "ARRAY JOIN Clause") - [JOIN Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-join "JOIN Clause") - [Processing of Empty or NULL Cells](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#processing-of-empty-or-null-cells "Processing of Empty or NULL Cells") - [WHERE Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#where-clause "WHERE Clause") - [PREWHERE Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#prewhere-clause "PREWHERE Clause") - [GROUP BY Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-group-by-clause "GROUP BY Clause") - [NULL processing](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#null-processing "NULL processing") - [WITH TOTALS Modifier](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#with-totals-modifier "WITH TOTALS Modifier") - [GROUP BY in External Memory](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-group-by-in-external-memory "GROUP BY in External Memory") - [LIMIT N BY Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#limit-n-by-clause "LIMIT N BY Clause") - [HAVING Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#having-clause "HAVING Clause") - [ORDER BY Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#order-by-clause "ORDER BY Clause") - [SELECT Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-clause "SELECT Clause") - [DISTINCT Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distinct "DISTINCT Clause") - [LIMIT Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#limit-clause "LIMIT Clause") - [UNION ALL Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#union-all-clause "UNION ALL Clause") - [INTO OUTFILE Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#into-outfile-clause "INTO OUTFILE Clause") - [FORMAT Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#format-clause "FORMAT Clause") - [IN Operators](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-in-operators "IN Operators") - [NULL processing](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#null-processing_1 "NULL processing") - [Distributed Subqueries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distributed-subqueries "Distributed Subqueries") - [Extreme Values](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#extreme-values "Extreme Values") - [Notes](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#notes "Notes") - [INSERT INTO](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/insert_into/ "INSERT INTO") - [CREATE](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/create/ "CREATE") - [ALTER](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/alter/ "ALTER") - [Other Kinds of Queries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/misc/ "Other Kinds of Queries") - Functions Functions - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/ "Introduction") - [Arithmetic](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/arithmetic_functions/ "Arithmetic") - [Comparison](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/comparison_functions/ "Comparison") - [Logical](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/logical_functions/ "Logical") - [Type Conversion](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/type_conversion_functions/ "Type Conversion") - [Working with Dates and Times](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/date_time_functions/ "Working with Dates and Times") - [Working with strings](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/string_functions/ "Working with strings") - [For Searching Strings](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/string_search_functions/ "For Searching Strings") - [For Replacing in Strings](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/string_replace_functions/ "For Replacing in Strings") - [Conditional](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/conditional_functions/ "Conditional ") - [Mathematical](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/math_functions/ "Mathematical") - [Rounding](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/rounding_functions/ "Rounding") - [Working with Arrays](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/array_functions/ "Working with Arrays") - [Splitting and Merging Strings and Arrays](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/splitting_merging_functions/ "Splitting and Merging Strings and Arrays") - [Bit](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/bit_functions/ "Bit") - [Bitmap functions](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/bitmap_functions/ "Bitmap functions") - [Hash](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/hash_functions/ "Hash") - [Generating Pseudo-Random Numbers](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/random_functions/ "Generating Pseudo-Random Numbers") - [Encoding](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/encoding_functions/ "Encoding") - [Working with UUID](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/uuid_functions/ "Working with UUID") - [Working with URLs](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/url_functions/ "Working with URLs") - [Working with IP Addresses](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/ip_address_functions/ "Working with IP Addresses") - [Working with JSON.](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/json_functions/ "Working with JSON.") - [Higher-Order](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/higher_order_functions/ "Higher-Order") - [Working with External Dictionaries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/ext_dict_functions/ "Working with External Dictionaries") - [Working with Yandex.Metrica Dictionaries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/ym_dict_functions/ "Working with Yandex.Metrica Dictionaries") - [Implementing the IN Operator](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/in_functions/ "Implementing the IN Operator") - [arrayJoin](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/array_join/ "arrayJoin") - [Working with geographical coordinates](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/geo/ "Working with geographical coordinates") - [Working with Nullable arguments](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/functions_for_nulls/ "Working with Nullable arguments") - [Other](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/functions/other_functions/ "Other") - Aggregate Functions Aggregate Functions - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/agg_functions/ "Introduction") - [Reference](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/agg_functions/reference/ "Reference") - [Aggregate function combinators](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/agg_functions/combinators/ "Aggregate function combinators") - [Parametric aggregate functions](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/agg_functions/parametric_functions/ "Parametric aggregate functions") - Table Functions Table Functions - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/table_functions/ "Introduction") - [file](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/table_functions/file/ "file") - [merge](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/table_functions/merge/ "merge") - [numbers](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/table_functions/numbers/ "numbers") - [remote](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/table_functions/remote/ "remote") - [url](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/table_functions/url/ "url") - [jdbc](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/table_functions/jdbc/ "jdbc") - Dictionaries Dictionaries - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/ "Introduction") - External Dictionaries External Dictionaries - [General Description](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/external_dicts/ "General Description") - [Configuring an External Dictionary](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/external_dicts_dict/ "Configuring an External Dictionary") - [Storing Dictionaries in Memory](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/external_dicts_dict_layout/ "Storing Dictionaries in Memory") - [Dictionary Updates](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/external_dicts_dict_lifetime/ "Dictionary Updates") - [Sources of External Dictionaries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/external_dicts_dict_sources/ "Sources of External Dictionaries") - [Dictionary Key and Fields](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/external_dicts_dict_structure/ "Dictionary Key and Fields") - [Internal Dictionaries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/internal_dicts/ "Internal Dictionaries") - [Operators](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/operators/ "Operators") - [General Syntax](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/syntax/ "General Syntax") - Operations Operations - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/ "Introduction") - [Requirements](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/requirements/ "Requirements") - [Monitoring](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/monitoring/ "Monitoring") - [Troubleshooting](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/troubleshooting/ "Troubleshooting") - [Usage Recommendations](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/tips/ "Usage Recommendations") - [ClickHouse Update](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/update/ "ClickHouse Update") - [Access Rights](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/access_rights/ "Access Rights") - [Data Backup](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/backup/ "Data Backup") - [Configuration Files](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/configuration_files/ "Configuration Files") - [Quotas](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/quotas/ "Quotas") - [System Tables](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/system_tables/ "System Tables") - Server Configuration Parameters Server Configuration Parameters - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/server_settings/ "Introduction") - [Server Settings](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/server_settings/settings/ "Server Settings") - Settings Settings - [Introduction](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/ "Introduction") - [Permissions for Queries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/permissions_for_queries/ "Permissions for Queries") - [Restrictions on Query Complexity](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/query_complexity/ "Restrictions on Query Complexity") - [Settings](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/settings/ "Settings") - [Settings Profiles](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/settings_profiles/ "Settings Profiles") - Utilities Utilities - [Overview](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/utils/ "Overview") - [clickhouse-copier](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/utils/clickhouse-copier/ "clickhouse-copier") - [clickhouse-local](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/utils/clickhouse-local/ "clickhouse-local") - F.A.Q. F.A.Q. - [General Questions](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/faq/general/ "General Questions") - Development Development - [Overview of ClickHouse Architecture](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/development/architecture/ "Overview of ClickHouse Architecture") - [How to Build ClickHouse on Linux](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/development/build/ "How to Build ClickHouse on Linux") - [How to Build ClickHouse on Mac OS X](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/development/build_osx/ "How to Build ClickHouse on Mac OS X") - [How to Write C++ code](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/development/style/ "How to Write C++ code") - [How to Run ClickHouse Tests](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/development/tests/ "How to Run ClickHouse Tests") - [Third-Party Libraries Used](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/development/contrib/ "Third-Party Libraries Used") - What's New What's New - [Roadmap](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/roadmap/ "Roadmap") - [Changelog](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/changelog/ "Changelog") - [Security Changelog](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/security_changelog/ "Security Changelog") - [Single page version](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/single/) - [PDF version](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/single/clickhouse_en.pdf) - [ClickHouse source code](https://github.com/yandex/ClickHouse/) Table of contents - [FROM Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#from-clause "FROM Clause") - [SAMPLE Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-clause "SAMPLE Clause") - [SAMPLE k](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-k "SAMPLE k") - [SAMPLE n](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-n "SAMPLE n") - [SAMPLE k OFFSET m](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-offset "SAMPLE k OFFSET m") - [ARRAY JOIN Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-array-join-clause "ARRAY JOIN Clause") - [JOIN Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-join "JOIN Clause") - [Processing of Empty or NULL Cells](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#processing-of-empty-or-null-cells "Processing of Empty or NULL Cells") - [WHERE Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#where-clause "WHERE Clause") - [PREWHERE Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#prewhere-clause "PREWHERE Clause") - [GROUP BY Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-group-by-clause "GROUP BY Clause") - [NULL processing](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#null-processing "NULL processing") - [WITH TOTALS Modifier](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#with-totals-modifier "WITH TOTALS Modifier") - [GROUP BY in External Memory](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-group-by-in-external-memory "GROUP BY in External Memory") - [LIMIT N BY Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#limit-n-by-clause "LIMIT N BY Clause") - [HAVING Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#having-clause "HAVING Clause") - [ORDER BY Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#order-by-clause "ORDER BY Clause") - [SELECT Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-clause "SELECT Clause") - [DISTINCT Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distinct "DISTINCT Clause") - [LIMIT Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#limit-clause "LIMIT Clause") - [UNION ALL Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#union-all-clause "UNION ALL Clause") - [INTO OUTFILE Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#into-outfile-clause "INTO OUTFILE Clause") - [FORMAT Clause](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#format-clause "FORMAT Clause") - [IN Operators](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-in-operators "IN Operators") - [NULL processing](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#null-processing_1 "NULL processing") - [Distributed Subqueries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distributed-subqueries "Distributed Subqueries") - [Extreme Values](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#extreme-values "Extreme Values") - [Notes](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#notes "Notes") [](https://github.com/yandex/ClickHouse/edit/master/docs/en/query_language/select.md "Edit this page") # SELECT Queries Syntax[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-queries-syntax "Permanent link") `SELECT` performs data retrieval. ``` SELECT [DISTINCT] expr_list [FROM [db.]table \| (subquery) \| table_function] [FINAL] [SAMPLE sample_coeff] [ARRAY JOIN ...] [GLOBAL] [ANY\|ALL] [INNER\|LEFT\|RIGHT\|FULL\|CROSS] [OUTER] JOIN (subquery)\|table USING columns_list [PREWHERE expr] [WHERE expr] [GROUP BY expr_list] [WITH TOTALS] [HAVING expr] [ORDER BY expr_list] [LIMIT [n, ]m] [UNION ALL ...] [INTO OUTFILE filename] [FORMAT format] [LIMIT n BY columns] ``` All the clauses are optional, except for the required list of expressions immediately after SELECT. The clauses below are described in almost the same order as in the query execution conveyor. If the query omits the `DISTINCT`, `GROUP BY` and `ORDER BY` clauses and the `IN` and `JOIN` subqueries, the query will be completely stream processed, using O(1) amount of RAM. Otherwise, the query might consume a lot of RAM if the appropriate restrictions are not specified: `max_memory_usage`, `max_rows_to_group_by`, `max_rows_to_sort`, `max_rows_in_distinct`, `max_bytes_in_distinct`, `max_rows_in_set`, `max_bytes_in_set`, `max_rows_in_join`, `max_bytes_in_join`, `max_bytes_before_external_sort`, `max_bytes_before_external_group_by`. For more information, see the section "Settings". It is possible to use external sorting (saving temporary tables to a disk) and external aggregation. `The system does not have "merge join"`. ### FROM Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#from-clause "Permanent link") If the FROM clause is omitted, data will be read from the `system.one` table. The 'system.one' table contains exactly one row (this table fulfills the same purpose as the DUAL table found in other DBMSs). The FROM clause specifies the table to read data from, or a subquery, or a table function; ARRAY JOIN and the regular JOIN may also be included (see below). Instead of a table, the SELECT subquery may be specified in brackets. In this case, the subquery processing pipeline will be built into the processing pipeline of an external query. In contrast to standard SQL, a synonym does not need to be specified after a subquery. For compatibility, it is possible to write 'AS name' after a subquery, but the specified name isn't used anywhere. A table function may be specified instead of a table. For more information, see the section "Table functions". To execute a query, all the columns listed in the query are extracted from the appropriate table. Any columns not needed for the external query are thrown out of the subqueries. If a query does not list any columns (for example, SELECT count() FROM t), some column is extracted from the table anyway (the smallest one is preferred), in order to calculate the number of rows. The FINAL modifier can be used only for a SELECT from a CollapsingMergeTree table. When you specify FINAL, data is selected fully "collapsed". Keep in mind that using FINAL leads to a selection that includes columns related to the primary key, in addition to the columns specified in the SELECT. Additionally, the query will be executed in a single stream, and data will be merged during query execution. This means that when using FINAL, the query is processed more slowly. In most cases, you should avoid using FINAL. For more information, see the section "CollapsingMergeTree engine". ### SAMPLE Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-clause "Permanent link") The `SAMPLE` clause allows for approximated query processing. Approximated query processing is only supported by the tables in the `MergeTree` family, and only if the sampling expression was specified during table creation (see [MergeTree engine](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/mergetree/)). The features of data sampling are listed below: - Data sampling is a deterministic mechanism. The result of the same `SELECT .. SAMPLE` query is always the same. - Sampling works consistently for different tables. For tables with a single sampling key, a sample with the same coefficient always selects the same subset of possible data. For example, a sample of user IDs takes rows with the same subset of all the possible user IDs from different tables. This allows using the sample in subqueries in the `IN` clause, as well as for manually correlating results of different queries with samples. - Sampling allows reading less data from a disk. Note that for this you must specify the sampling key correctly. For more details see [Creating a MergeTree Table](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/mergetree/#table_engine-mergetree-creating-a-table). The `SAMPLE` clause can be specified in several ways: - `SAMPLE k`, where `k` is a decimal number from 0 to 1. The query is executed on `k` percent of data. For example, `SAMPLE 0.1` runs the query on 10% of data. [Read more](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-k) - `SAMPLE n`, where `n` is a sufficiently large integer. The query is executed on a sample of at least `n` rows (but not significantly more than this). For example, `SAMPLE 10000000` runs the query on a minimum of 10,000,000 rows. [Read more](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-n) - `SAMPLE k OFFSET m` where `k` and `m` are numbers from 0 to 1. The query is executed on a sample of `k` percent of the data. The data used for the sample is offset by `m` percent. [Read more](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-offset) #### SAMPLE k[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-k "Permanent link") In a `SAMPLE k` clause, `k` is a percent amount of data that the sample is taken from. The example is shown below: ``` SELECT Title, count() * 10 AS PageViews FROM hits_distributed SAMPLE 0.1 WHERE CounterID = 34 GROUP BY Title ORDER BY PageViews DESC LIMIT 1000 ``` In this example, the query is executed on a sample from 0.1 (10%) of data. Values of aggregate functions are not corrected automatically, so to get an approximate result, the value 'count()' is manually multiplied by 10. #### SAMPLE n[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-n "Permanent link") In this case, the query is executed on a sample of at least `n` rows, where `n` is a sufficiently large integer. For example, `SAMPLE 10000000`. Since the minimum unit for data reading is one granule (its size is set by the `index_granularity` setting), it makes sense to set a sample that is much larger than the size of the granule. When using the `SAMPLE n` clause, the relative coefficient is calculated dynamically. Since you do not know which relative percent of data was processed, you do not know the coefficient the aggregate functions should be multiplied by (for example, you do not know if the `SAMPLE 1000000` was taken from a set of 10,000,000 rows or from a set of 1,000,000,000 rows). In this case, use the `_sample_factor` column to get the approximate result. The `_sample_factor` is a virtual column that ClickHouse stores relative coefficients in. This column is created automatically when you create a table with the specified sampling key. The usage example is shown below: ``` SELECT sum(Duration * _sample_factor) FROM visits SAMPLE 10000000 ``` If you need to get the approximate count of rows in a `SELECT .. SAMPLE n` query, get the sum() of `_sample_factor` column instead of counting `count(column * _sample_factor)` value. For example: ``` SELECT sum(_sample_factor) FROM visits SAMPLE 10000000 ``` Note that to calculate the average in a `SELECT .. SAMPLE n` query, you do not need to use `_sample_factor` column: ``` SELECT avg(Duration) FROM visits SAMPLE 10000000 ``` #### SAMPLE k OFFSET m[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-offset "Permanent link") You can specify the `SAMPLE k OFFSET m` clause, where `k` and `m` are numbers from 0 to 1. Examples are shown below. Example 1. ``` SAMPLE 1/10 ``` In this example, the sample is the 1/10th of all data: `[++------------------]` Example 2. ``` SAMPLE 1/10 OFFSET 1/2 ``` Here, the sample of 10% is taken from the second half of data. `[----------++--------]` ### ARRAY JOIN Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-array-join-clause "Permanent link") Allows executing JOIN with an array or nested data structure. The intent is similar to the 'arrayJoin' function, but its functionality is broader. `ARRAY JOIN` is essentially `INNER JOIN` with an array. Example: ``` :) CREATE TABLE arrays_test (s String, arr Array(UInt8)) ENGINE = Memory CREATE TABLE arrays_test ( s String, arr Array(UInt8) ) ENGINE = Memory Ok. 0 rows in set. Elapsed: 0.001 sec. :) INSERT INTO arrays_test VALUES ('Hello', [1,2]), ('World', [3,4,5]), ('Goodbye', []) INSERT INTO arrays_test VALUES Ok. 3 rows in set. Elapsed: 0.001 sec. :) SELECT * FROM arrays_test SELECT * FROM arrays_test ┌─s───────┬─arr─────┐ │ Hello │ [1,2] │ │ World │ [3,4,5] │ │ Goodbye │ [] │ └─────────┴─────────┘ 3 rows in set. Elapsed: 0.001 sec. :) SELECT s, arr FROM arrays_test ARRAY JOIN arr SELECT s, arr FROM arrays_test ARRAY JOIN arr ┌─s─────┬─arr─┐ │ Hello │ 1 │ │ Hello │ 2 │ │ World │ 3 │ │ World │ 4 │ │ World │ 5 │ └───────┴─────┘ 5 rows in set. Elapsed: 0.001 sec. ``` An alias can be specified for an array in the ARRAY JOIN clause. In this case, an array item can be accessed by this alias, but the array itself by the original name. Example: ``` :) SELECT s, arr, a FROM arrays_test ARRAY JOIN arr AS a SELECT s, arr, a FROM arrays_test ARRAY JOIN arr AS a ┌─s─────┬─arr─────┬─a─┐ │ Hello │ [1,2] │ 1 │ │ Hello │ [1,2] │ 2 │ │ World │ [3,4,5] │ 3 │ │ World │ [3,4,5] │ 4 │ │ World │ [3,4,5] │ 5 │ └───────┴─────────┴───┘ 5 rows in set. Elapsed: 0.001 sec. ``` Multiple arrays of the same size can be comma-separated in the ARRAY JOIN clause. In this case, JOIN is performed with them simultaneously (the direct sum, not the direct product). Example: ``` :) SELECT s, arr, a, num, mapped FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(x -> x + 1, arr) AS mapped SELECT s, arr, a, num, mapped FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(lambda(tuple(x), plus(x, 1)), arr) AS mapped ┌─s─────┬─arr─────┬─a─┬─num─┬─mapped─┐ │ Hello │ [1,2] │ 1 │ 1 │ 2 │ │ Hello │ [1,2] │ 2 │ 2 │ 3 │ │ World │ [3,4,5] │ 3 │ 1 │ 4 │ │ World │ [3,4,5] │ 4 │ 2 │ 5 │ │ World │ [3,4,5] │ 5 │ 3 │ 6 │ └───────┴─────────┴───┴─────┴────────┘ 5 rows in set. Elapsed: 0.002 sec. :) SELECT s, arr, a, num, arrayEnumerate(arr) FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num SELECT s, arr, a, num, arrayEnumerate(arr) FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num ┌─s─────┬─arr─────┬─a─┬─num─┬─arrayEnumerate(arr)─┐ │ Hello │ [1,2] │ 1 │ 1 │ [1,2] │ │ Hello │ [1,2] │ 2 │ 2 │ [1,2] │ │ World │ [3,4,5] │ 3 │ 1 │ [1,2,3] │ │ World │ [3,4,5] │ 4 │ 2 │ [1,2,3] │ │ World │ [3,4,5] │ 5 │ 3 │ [1,2,3] │ └───────┴─────────┴───┴─────┴─────────────────────┘ 5 rows in set. Elapsed: 0.002 sec. ``` ARRAY JOIN also works with nested data structures. Example: ``` :) CREATE TABLE nested_test (s String, nest Nested(x UInt8, y UInt32)) ENGINE = Memory CREATE TABLE nested_test ( s String, nest Nested( x UInt8, y UInt32) ) ENGINE = Memory Ok. 0 rows in set. Elapsed: 0.006 sec. :) INSERT INTO nested_test VALUES ('Hello', [1,2], [10,20]), ('World', [3,4,5], [30,40,50]), ('Goodbye', [], []) INSERT INTO nested_test VALUES Ok. 3 rows in set. Elapsed: 0.001 sec. :) SELECT * FROM nested_test SELECT * FROM nested_test ┌─s───────┬─nest.x──┬─nest.y─────┐ │ Hello │ [1,2] │ [10,20] │ │ World │ [3,4,5] │ [30,40,50] │ │ Goodbye │ [] │ [] │ └─────────┴─────────┴────────────┘ 3 rows in set. Elapsed: 0.001 sec. :) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest SELECT s, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN nest ┌─s─────┬─nest.x─┬─nest.y─┐ │ Hello │ 1 │ 10 │ │ Hello │ 2 │ 20 │ │ World │ 3 │ 30 │ │ World │ 4 │ 40 │ │ World │ 5 │ 50 │ └───────┴────────┴────────┘ 5 rows in set. Elapsed: 0.001 sec. ``` When specifying names of nested data structures in ARRAY JOIN, the meaning is the same as ARRAY JOIN with all the array elements that it consists of. Example: ``` :) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x, nest.y SELECT s, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN `nest.x`, `nest.y` ┌─s─────┬─nest.x─┬─nest.y─┐ │ Hello │ 1 │ 10 │ │ Hello │ 2 │ 20 │ │ World │ 3 │ 30 │ │ World │ 4 │ 40 │ │ World │ 5 │ 50 │ └───────┴────────┴────────┘ 5 rows in set. Elapsed: 0.001 sec. ``` This variation also makes sense: ``` :) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x SELECT s, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN `nest.x` ┌─s─────┬─nest.x─┬─nest.y─────┐ │ Hello │ 1 │ [10,20] │ │ Hello │ 2 │ [10,20] │ │ World │ 3 │ [30,40,50] │ │ World │ 4 │ [30,40,50] │ │ World │ 5 │ [30,40,50] │ └───────┴────────┴────────────┘ 5 rows in set. Elapsed: 0.001 sec. ``` An alias may be used for a nested data structure, in order to select either the JOIN result or the source array. Example: ``` :) SELECT s, n.x, n.y, nest.x, nest.y FROM nested_test ARRAY JOIN nest AS n SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN nest AS n ┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┐ │ Hello │ 1 │ 10 │ [1,2] │ [10,20] │ │ Hello │ 2 │ 20 │ [1,2] │ [10,20] │ │ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │ │ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │ │ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │ └───────┴─────┴─────┴─────────┴────────────┘ 5 rows in set. Elapsed: 0.001 sec. ``` Example of using the arrayEnumerate function: ``` :) SELECT s, n.x, n.y, nest.x, nest.y, num FROM nested_test ARRAY JOIN nest AS n, arrayEnumerate(nest.x) AS num SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`, num FROM nested_test ARRAY JOIN nest AS n, arrayEnumerate(`nest.x`) AS num ┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┬─num─┐ │ Hello │ 1 │ 10 │ [1,2] │ [10,20] │ 1 │ │ Hello │ 2 │ 20 │ [1,2] │ [10,20] │ 2 │ │ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │ 1 │ │ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │ 2 │ │ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │ 3 │ └───────┴─────┴─────┴─────────┴────────────┴─────┘ 5 rows in set. Elapsed: 0.002 sec. ``` The query can only specify a single ARRAY JOIN clause. The corresponding conversion can be performed before the WHERE/PREWHERE clause (if its result is needed in this clause), or after completing WHERE/PREWHERE (to reduce the volume of calculations). ### JOIN Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-join "Permanent link") Joins the data in the normal [SQL JOIN](https://en.wikipedia.org/wiki/Join_\(SQL\)) sense. Note Not related to [ARRAY JOIN](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-array-join-clause). ``` SELECT <expr_list> FROM <left_subquery> [GLOBAL] [ANY\|ALL] [INNER\|LEFT\|RIGHT\|FULL\|CROSS] [OUTER] JOIN <right_subquery> (ON <expr_list>)\|(USING <column_list>) ... ``` The table names can be specified instead of `<left_subquery>` and `<right_subquery>`. This is equivalent to the `SELECT * FROM table` subquery, except in a special case when the table has the [Join](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/join/) engine – an array prepared for joining. Supported types of `JOIN` - `INNER JOIN` (or `JOIN`) - `LEFT JOIN` (or `LEFT OUTER JOIN`) - `RIGHT JOIN` (or `RIGHT OUTER JOIN`) - `FULL JOIN` (or `FULL OUTER JOIN`) - `CROSS JOIN` (or `,` ) See standard [SQL JOIN](https://en.wikipedia.org/wiki/Join_\(SQL\)) description. ANY or ALL strictness If `ALL` is specified and the right table has several matching rows, the data will be multiplied by the number of these rows. This is the normal `JOIN` behavior for standard SQL. If `ANY` is specified and the right table has several matching rows, only the first one found is joined. If the right table has only one matching row, the results of `ANY` and `ALL` are the same. To set the default strictness value, use the session configuration parameter [join\_default\_strictness](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/settings/#settings-join_default_strictness). GLOBAL JOIN When using a normal `JOIN`, the query is sent to remote servers. Subqueries are run on each of them in order to make the right table, and the join is performed with this table. In other words, the right table is formed on each server separately. When using `GLOBAL ... JOIN`, first the requestor server runs a subquery to calculate the right table. This temporary table is passed to each remote server, and queries are run on them using the temporary data that was transmitted. Be careful when using `GLOBAL`. For more information, see the section [Distributed subqueries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distributed-subqueries). Usage Recommendations All columns that are not needed for the `JOIN` are deleted from the subquery. When running a `JOIN`, there is no optimization of the order of execution in relation to other stages of the query. The join (a search in the right table) is run before filtering in `WHERE` and before aggregation. In order to explicitly set the processing order, we recommend running a `JOIN` subquery with a subquery. Example: ``` SELECT CounterID, hits, visits FROM ( SELECT CounterID, count() AS hits FROM test.hits GROUP BY CounterID ) ANY LEFT JOIN ( SELECT CounterID, sum(Sign) AS visits FROM test.visits GROUP BY CounterID ) USING CounterID ORDER BY hits DESC LIMIT 10 ``` ``` ┌─CounterID─┬───hits─┬─visits─┐ │ 1143050 │ 523264 │ 13665 │ │ 731962 │ 475698 │ 102716 │ │ 722545 │ 337212 │ 108187 │ │ 722889 │ 252197 │ 10547 │ │ 2237260 │ 196036 │ 9522 │ │ 23057320 │ 147211 │ 7689 │ │ 722818 │ 90109 │ 17847 │ │ 48221 │ 85379 │ 4652 │ │ 19762435 │ 77807 │ 7026 │ │ 722884 │ 77492 │ 11056 │ └───────────┴────────┴────────┘ ``` Subqueries don't allow you to set names or use them for referencing a column from a specific subquery. The columns specified in `USING` must have the same names in both subqueries, and the other columns must be named differently. You can use aliases to change the names of columns in subqueries (the example uses the aliases 'hits' and 'visits'). The `USING` clause specifies one or more columns to join, which establishes the equality of these columns. The list of columns is set without brackets. More complex join conditions are not supported. The right table (the subquery result) resides in RAM. If there isn't enough memory, you can't run a `JOIN`. Each time a query is run with the same `JOIN`, the subquery is run again because the result is not cached. To avoid this, use the special [Join](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/join/) table engine, which is a prepared array for joining that is always in RAM. In some cases, it is more efficient to use `IN` instead of `JOIN`. Among the various types of `JOIN`, the most efficient is `ANY LEFT JOIN`, then `ANY INNER JOIN`. The least efficient are `ALL LEFT JOIN` and `ALL INNER JOIN`. If you need a `JOIN` for joining with dimension tables (these are relatively small tables that contain dimension properties, such as names for advertising campaigns), a `JOIN` might not be very convenient due to the bulky syntax and the fact that the right table is re-accessed for every query. For such cases, there is an "external dictionaries" feature that you should use instead of `JOIN`. For more information, see the section [External dictionaries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/external_dicts/). #### Processing of Empty or NULL Cells[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#processing-of-empty-or-null-cells "Permanent link") While joining tables, the empty cells may appear. The setting [join\_use\_nulls](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/settings/#settings-join_use_nulls) define how ClickHouse fills these cells. If the `JOIN` keys are [Nullable](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/nullable/) fields, the rows where at least one of the keys has the value [NULL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/syntax/#null-literal) are not joined. ### WHERE Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#where-clause "Permanent link") If there is a WHERE clause, it must contain an expression with the UInt8 type. This is usually an expression with comparison and logical operators. This expression will be used for filtering data before all other transformations. If indexes are supported by the database table engine, the expression is evaluated on the ability to use indexes. ### PREWHERE Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#prewhere-clause "Permanent link") This clause has the same meaning as the WHERE clause. The difference is in which data is read from the table. When using PREWHERE, first only the columns necessary for executing PREWHERE are read. Then the other columns are read that are needed for running the query, but only those blocks where the PREWHERE expression is true. It makes sense to use PREWHERE if there are filtration conditions that are used by a minority of the columns in the query, but that provide strong data filtration. This reduces the volume of data to read. For example, it is useful to write PREWHERE for queries that extract a large number of columns, but that only have filtration for a few columns. PREWHERE is only supported by tables from the `MergeTree` family. A query may simultaneously specify PREWHERE and WHERE. In this case, PREWHERE precedes WHERE. If the 'optimize\_move\_to\_prewhere' setting is set to 1 and PREWHERE is omitted, the system uses heuristics to automatically move parts of expressions from WHERE to PREWHERE. ### GROUP BY Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-group-by-clause "Permanent link") This is one of the most important parts of a column-oriented DBMS. If there is a GROUP BY clause, it must contain a list of expressions. Each expression will be referred to here as a "key". All the expressions in the SELECT, HAVING, and ORDER BY clauses must be calculated from keys or from aggregate functions. In other words, each column selected from the table must be used either in keys or inside aggregate functions. If a query contains only table columns inside aggregate functions, the GROUP BY clause can be omitted, and aggregation by an empty set of keys is assumed. Example: ``` SELECT count(), median(FetchTiming > 60 ? 60 : FetchTiming), count() - sum(Refresh) FROM hits ``` However, in contrast to standard SQL, if the table doesn't have any rows (either there aren't any at all, or there aren't any after using WHERE to filter), an empty result is returned, and not the result from one of the rows containing the initial values of aggregate functions. As opposed to MySQL (and conforming to standard SQL), you can't get some value of some column that is not in a key or aggregate function (except constant expressions). To work around this, you can use the 'any' aggregate function (get the first encountered value) or 'min/max'. Example: ``` SELECT domainWithoutWWW(URL) AS domain, count(), any(Title) AS title -- getting the first occurred page header for each domain. FROM hits GROUP BY domain ``` For every different key value encountered, GROUP BY calculates a set of aggregate function values. GROUP BY is not supported for array columns. A constant can't be specified as arguments for aggregate functions. Example: sum(1). Instead of this, you can get rid of the constant. Example: `count()`. #### NULL processing[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#null-processing "Permanent link") For grouping, ClickHouse interprets [NULL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/syntax/) as a value, and `NULL=NULL`. Here's an example to show what this means. Assume you have this table: ``` ┌─x─┬────y─┐ │ 1 │ 2 │ │ 2 │ ᴺᵁᴸᴸ │ │ 3 │ 2 │ │ 3 │ 3 │ │ 3 │ ᴺᵁᴸᴸ │ └───┴──────┘ ``` The query `SELECT sum(x), y FROM t_null_big GROUP BY y` results in: ``` ┌─sum(x)─┬────y─┐ │ 4 │ 2 │ │ 3 │ 3 │ │ 5 │ ᴺᵁᴸᴸ │ └────────┴──────┘ ``` You can see that `GROUP BY` for `У = NULL` summed up `x`, as if `NULL` is this value. If you pass several keys to `GROUP BY`, the result will give you all the combinations of the selection, as if `NULL` were a specific value. #### WITH TOTALS Modifier[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#with-totals-modifier "Permanent link") If the WITH TOTALS modifier is specified, another row will be calculated. This row will have key columns containing default values (zeros or empty lines), and columns of aggregate functions with the values calculated across all the rows (the "total" values). This extra row is output in JSON\, TabSeparated\, and Pretty\ formats, separately from the other rows. In the other formats, this row is not output. In JSON\* formats, this row is output as a separate 'totals' field. In TabSeparated\* formats, the row comes after the main result, preceded by an empty row (after the other data). In Pretty\* formats, the row is output as a separate table after the main result. `WITH TOTALS` can be run in different ways when HAVING is present. The behavior depends on the 'totals\_mode' setting. By default, `totals_mode = 'before_having'`. In this case, 'totals' is calculated across all rows, including the ones that don't pass through HAVING and 'max\_rows\_to\_group\_by'. The other alternatives include only the rows that pass through HAVING in 'totals', and behave differently with the setting `max_rows_to_group_by` and `group_by_overflow_mode = 'any'`. `after_having_exclusive` – Don't include rows that didn't pass through `max_rows_to_group_by`. In other words, 'totals' will have less than or the same number of rows as it would if `max_rows_to_group_by` were omitted. `after_having_inclusive` – Include all the rows that didn't pass through 'max\_rows\_to\_group\_by' in 'totals'. In other words, 'totals' will have more than or the same number of rows as it would if `max_rows_to_group_by` were omitted. `after_having_auto` – Count the number of rows that passed through HAVING. If it is more than a certain amount (by default, 50%), include all the rows that didn't pass through 'max\_rows\_to\_group\_by' in 'totals'. Otherwise, do not include them. `totals_auto_threshold` – By default, 0.5. The coefficient for `after_having_auto`. If `max_rows_to_group_by` and `group_by_overflow_mode = 'any'` are not used, all variations of `after_having` are the same, and you can use any of them (for example, `after_having_auto`). You can use WITH TOTALS in subqueries, including subqueries in the JOIN clause (in this case, the respective total values are combined). #### GROUP BY in External Memory[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-group-by-in-external-memory "Permanent link") You can enable dumping temporary data to the disk to restrict memory usage during GROUP BY. The `max_bytes_before_external_group_by` setting determines the threshold RAM consumption for dumping GROUP BY temporary data to the file system. If set to 0 (the default), it is disabled. When using `max_bytes_before_external_group_by`, we recommend that you set max\_memory\_usage about twice as high. This is necessary because there are two stages to aggregation: reading the date and forming intermediate data (1) and merging the intermediate data (2). Dumping data to the file system can only occur during stage 1. If the temporary data wasn't dumped, then stage 2 might require up to the same amount of memory as in stage 1. For example, if `max_memory_usage` was set to 10000000000 and you want to use external aggregation, it makes sense to set `max_bytes_before_external_group_by` to 10000000000, and max\_memory\_usage to 20000000000. When external aggregation is triggered (if there was at least one dump of temporary data), maximum consumption of RAM is only slightly more than `max_bytes_before_external_group_by`. With distributed query processing, external aggregation is performed on remote servers. In order for the requestor server to use only a small amount of RAM, set `distributed_aggregation_memory_efficient` to 1. When merging data flushed to the disk, as well as when merging results from remote servers when the `distributed_aggregation_memory_efficient` setting is enabled, consumes up to 1/256 \* the number of threads from the total amount of RAM. When external aggregation is enabled, if there was less than `max_bytes_before_external_group_by` of data (i.e. data was not flushed), the query runs just as fast as without external aggregation. If any temporary data was flushed, the run time will be several times longer (approximately three times). If you have an ORDER BY with a small LIMIT after GROUP BY, then the ORDER BY CLAUSE will not use significant amounts of RAM. But if the ORDER BY doesn't have LIMIT, don't forget to enable external sorting (`max_bytes_before_external_sort`). ### LIMIT N BY Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#limit-n-by-clause "Permanent link") LIMIT N BY COLUMNS selects the top N rows for each group of COLUMNS. LIMIT N BY is not related to LIMIT; they can both be used in the same query. The key for LIMIT N BY can contain any number of columns or expressions. Example: ``` SELECT domainWithoutWWW(URL) AS domain, domainWithoutWWW(REFERRER_URL) AS referrer, device_type, count() cnt FROM hits GROUP BY domain, referrer, device_type ORDER BY cnt DESC LIMIT 5 BY domain, device_type LIMIT 100 ``` The query will select the top 5 referrers for each `domain, device_type` pair, but not more than 100 rows (`LIMIT n BY + LIMIT`). ### HAVING Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#having-clause "Permanent link") Allows filtering the result received after GROUP BY, similar to the WHERE clause. WHERE and HAVING differ in that WHERE is performed before aggregation (GROUP BY), while HAVING is performed after it. If aggregation is not performed, HAVING can't be used. ### ORDER BY Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#order-by-clause "Permanent link") The ORDER BY clause contains a list of expressions, which can each be assigned DESC or ASC (the sorting direction). If the direction is not specified, ASC is assumed. ASC is sorted in ascending order, and DESC in descending order. The sorting direction applies to a single expression, not to the entire list. Example: `ORDER BY Visits DESC, SearchPhrase` For sorting by String values, you can specify collation (comparison). Example: `ORDER BY SearchPhrase COLLATE 'tr'` - for sorting by keyword in ascending order, using the Turkish alphabet, case insensitive, assuming that strings are UTF-8 encoded. COLLATE can be specified or not for each expression in ORDER BY independently. If ASC or DESC is specified, COLLATE is specified after it. When using COLLATE, sorting is always case-insensitive. We only recommend using COLLATE for final sorting of a small number of rows, since sorting with COLLATE is less efficient than normal sorting by bytes. Rows that have identical values for the list of sorting expressions are output in an arbitrary order, which can also be nondeterministic (different each time). If the ORDER BY clause is omitted, the order of the rows is also undefined, and may be nondeterministic as well. `NaN` and `NULL` sorting order: - With the modifier `NULLS FIRST` — First `NULL`, then `NaN`, then other values. - With the modifier `NULLS LAST` — First the values, then `NaN`, then `NULL`. - Default — The same as with the `NULLS LAST` modifier. Example: For the table ``` ┌─x─┬────y─┐ │ 1 │ ᴺᵁᴸᴸ │ │ 2 │ 2 │ │ 1 │ nan │ │ 2 │ 2 │ │ 3 │ 4 │ │ 5 │ 6 │ │ 6 │ nan │ │ 7 │ ᴺᵁᴸᴸ │ │ 6 │ 7 │ │ 8 │ 9 │ └───┴──────┘ ``` Run the query `SELECT * FROM t_null_nan ORDER BY y NULLS FIRST` to get: ``` ┌─x─┬────y─┐ │ 1 │ ᴺᵁᴸᴸ │ │ 7 │ ᴺᵁᴸᴸ │ │ 1 │ nan │ │ 6 │ nan │ │ 2 │ 2 │ │ 2 │ 2 │ │ 3 │ 4 │ │ 5 │ 6 │ │ 6 │ 7 │ │ 8 │ 9 │ └───┴──────┘ ``` When floating point numbers are sorted, NaNs are separate from the other values. Regardless of the sorting order, NaNs come at the end. In other words, for ascending sorting they are placed as if they are larger than all the other numbers, while for descending sorting they are placed as if they are smaller than the rest. Less RAM is used if a small enough LIMIT is specified in addition to ORDER BY. Otherwise, the amount of memory spent is proportional to the volume of data for sorting. For distributed query processing, if GROUP BY is omitted, sorting is partially done on remote servers, and the results are merged on the requestor server. This means that for distributed sorting, the volume of data to sort can be greater than the amount of memory on a single server. If there is not enough RAM, it is possible to perform sorting in external memory (creating temporary files on a disk). Use the setting `max_bytes_before_external_sort` for this purpose. If it is set to 0 (the default), external sorting is disabled. If it is enabled, when the volume of data to sort reaches the specified number of bytes, the collected data is sorted and dumped into a temporary file. After all data is read, all the sorted files are merged and the results are output. Files are written to the /var/lib/clickhouse/tmp/ directory in the config (by default, but you can use the 'tmp\_path' parameter to change this setting). Running a query may use more memory than 'max\_bytes\_before\_external\_sort'. For this reason, this setting must have a value significantly smaller than 'max\_memory\_usage'. As an example, if your server has 128 GB of RAM and you need to run a single query, set 'max\_memory\_usage' to 100 GB, and 'max\_bytes\_before\_external\_sort' to 80 GB. External sorting works much less effectively than sorting in RAM. ### SELECT Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-clause "Permanent link") The expressions specified in the SELECT clause are analyzed after the calculations for all the clauses listed above are completed. More specifically, expressions are analyzed that are above the aggregate functions, if there are any aggregate functions. The aggregate functions and everything below them are calculated during aggregation (GROUP BY). These expressions work as if they are applied to separate rows in the result. ### DISTINCT Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distinct "Permanent link") If DISTINCT is specified, only a single row will remain out of all the sets of fully matching rows in the result. The result will be the same as if GROUP BY were specified across all the fields specified in SELECT without aggregate functions. But there are several differences from GROUP BY: - DISTINCT can be applied together with GROUP BY. - When ORDER BY is omitted and LIMIT is defined, the query stops running immediately after the required number of different rows has been read. - Data blocks are output as they are processed, without waiting for the entire query to finish running. DISTINCT is not supported if SELECT has at least one array column. `DISTINCT` works with [NULL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/syntax/) as if `NULL` were a specific value, and `NULL=NULL`. In other words, in the `DISTINCT` results, different combinations with `NULL` only occur once. ### LIMIT Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#limit-clause "Permanent link") `LIMIT m` allows you to select the first `m` rows from the result. `LIMIT n`, m allows you to select the first `m` rows from the result after skipping the first `n` rows. `n` and `m` must be non-negative integers. If there isn't an ORDER BY clause that explicitly sorts results, the result may be arbitrary and nondeterministic. ### UNION ALL Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#union-all-clause "Permanent link") You can use UNION ALL to combine any number of queries. Example: ``` SELECT CounterID, 1 AS table, toInt64(count()) AS c FROM test.hits GROUP BY CounterID UNION ALL SELECT CounterID, 2 AS table, sum(Sign) AS c FROM test.visits GROUP BY CounterID HAVING c > 0 ``` Only UNION ALL is supported. The regular UNION (UNION DISTINCT) is not supported. If you need UNION DISTINCT, you can write SELECT DISTINCT from a subquery containing UNION ALL. Queries that are parts of UNION ALL can be run simultaneously, and their results can be mixed together. The structure of results (the number and type of columns) must match for the queries. But the column names can differ. In this case, the column names for the final result will be taken from the first query. Type casting is performed for unions. For example, if two queries being combined have the same field with non-`Nullable` and `Nullable` types from a compatible type, the resulting `UNION ALL` has a `Nullable` type field. Queries that are parts of UNION ALL can't be enclosed in brackets. ORDER BY and LIMIT are applied to separate queries, not to the final result. If you need to apply a conversion to the final result, you can put all the queries with UNION ALL in a subquery in the FROM clause. ### INTO OUTFILE Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#into-outfile-clause "Permanent link") Add the `INTO OUTFILE filename` clause (where filename is a string literal) to redirect query output to the specified file. In contrast to MySQL, the file is created on the client side. The query will fail if a file with the same filename already exists. This functionality is available in the command-line client and clickhouse-local (a query sent via HTTP interface will fail). The default output format is TabSeparated (the same as in the command-line client batch mode). ### FORMAT Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#format-clause "Permanent link") Specify 'FORMAT format' to get data in any specified format. You can use this for convenience, or for creating dumps. For more information, see the section "Formats". If the FORMAT clause is omitted, the default format is used, which depends on both the settings and the interface used for accessing the DB. For the HTTP interface and the command-line client in batch mode, the default format is TabSeparated. For the command-line client in interactive mode, the default format is PrettyCompact (it has attractive and compact tables). When using the command-line client, data is passed to the client in an internal efficient format. The client independently interprets the FORMAT clause of the query and formats the data itself (thus relieving the network and the server from the load). ### IN Operators[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-in-operators "Permanent link") The `IN`, `NOT IN`, `GLOBAL IN`, and `GLOBAL NOT IN` operators are covered separately, since their functionality is quite rich. The left side of the operator is either a single column or a tuple. Examples: ``` SELECT UserID IN (123, 456) FROM ... SELECT (CounterID, UserID) IN ((34, 123), (101500, 456)) FROM ... ``` If the left side is a single column that is in the index, and the right side is a set of constants, the system uses the index for processing the query. Don't list too many values explicitly (i.e. millions). If a data set is large, put it in a temporary table (for example, see the section "External data for query processing"), then use a subquery. The right side of the operator can be a set of constant expressions, a set of tuples with constant expressions (shown in the examples above), or the name of a database table or SELECT subquery in brackets. If the right side of the operator is the name of a table (for example, `UserID IN users`), this is equivalent to the subquery `UserID IN (SELECT * FROM users)`. Use this when working with external data that is sent along with the query. For example, the query can be sent together with a set of user IDs loaded to the 'users' temporary table, which should be filtered. If the right side of the operator is a table name that has the Set engine (a prepared data set that is always in RAM), the data set will not be created over again for each query. The subquery may specify more than one column for filtering tuples. Example: ``` SELECT (CounterID, UserID) IN (SELECT CounterID, UserID FROM ...) FROM ... ``` The columns to the left and right of the IN operator should have the same type. The IN operator and subquery may occur in any part of the query, including in aggregate functions and lambda functions. Example: ``` SELECT EventDate, avg(UserID IN ( SELECT UserID FROM test.hits WHERE EventDate = toDate('2014-03-17') )) AS ratio FROM test.hits GROUP BY EventDate ORDER BY EventDate ASC ``` ``` ┌──EventDate─┬────ratio─┐ │ 2014-03-17 │ 1 │ │ 2014-03-18 │ 0.807696 │ │ 2014-03-19 │ 0.755406 │ │ 2014-03-20 │ 0.723218 │ │ 2014-03-21 │ 0.697021 │ │ 2014-03-22 │ 0.647851 │ │ 2014-03-23 │ 0.648416 │ └────────────┴──────────┘ ``` For each day after March 17th, count the percentage of pageviews made by users who visited the site on March 17th. A subquery in the IN clause is always run just one time on a single server. There are no dependent subqueries. #### NULL processing[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#null-processing_1 "Permanent link") During request processing, the IN operator assumes that the result of an operation with [NULL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/syntax/) is always equal to `0`, regardless of whether `NULL` is on the right or left side of the operator. `NULL` values are not included in any dataset, do not correspond to each other and cannot be compared. Here is an example with the `t_null` table: ``` ┌─x─┬────y─┐ │ 1 │ ᴺᵁᴸᴸ │ │ 2 │ 3 │ └───┴──────┘ ``` Running the query `SELECT x FROM t_null WHERE y IN (NULL,3)` gives you the following result: ``` ┌─x─┐ │ 2 │ └───┘ ``` You can see that the row in which `y = NULL` is thrown out of the query results. This is because ClickHouse can't decide whether `NULL` is included in the `(NULL,3)` set, returns `0` as the result of the operation, and `SELECT` excludes this row from the final output. ``` SELECT y IN (NULL, 3) FROM t_null ┌─in(y, tuple(NULL, 3))─┐ │ 0 │ │ 1 │ └───────────────────────┘ ``` #### Distributed Subqueries[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distributed-subqueries "Permanent link") There are two options for IN-s with subqueries (similar to JOINs): normal `IN` / `JOIN` and `GLOBAL IN` / `GLOBAL JOIN`. They differ in how they are run for distributed query processing. Attention Remember that the algorithms described below may work differently depending on the [settings](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/settings/) `distributed_product_mode` setting. When using the regular IN, the query is sent to remote servers, and each of them runs the subqueries in the `IN` or `JOIN` clause. When using `GLOBAL IN` / `GLOBAL JOINs`, first all the subqueries are run for `GLOBAL IN` / `GLOBAL JOINs`, and the results are collected in temporary tables. Then the temporary tables are sent to each remote server, where the queries are run using this temporary data. For a non-distributed query, use the regular `IN` / `JOIN`. Be careful when using subqueries in the `IN` / `JOIN` clauses for distributed query processing. Let's look at some examples. Assume that each server in the cluster has a normal local\_table. Each server also has a distributed\_table table with the Distributed type, which looks at all the servers in the cluster. For a query to the distributed\_table, the query will be sent to all the remote servers and run on them using the local\_table. For example, the query ``` SELECT uniq(UserID) FROM distributed_table ``` will be sent to all remote servers as ``` SELECT uniq(UserID) FROM local_table ``` and run on each of them in parallel, until it reaches the stage where intermediate results can be combined. Then the intermediate results will be returned to the requestor server and merged on it, and the final result will be sent to the client. Now let's examine a query with IN: ``` SELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34) ``` - Calculation of the intersection of audiences of two sites. This query will be sent to all remote servers as ``` SELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34) ``` In other words, the data set in the IN clause will be collected on each server independently, only across the data that is stored locally on each of the servers. This will work correctly and optimally if you are prepared for this case and have spread data across the cluster servers such that the data for a single UserID resides entirely on a single server. In this case, all the necessary data will be available locally on each server. Otherwise, the result will be inaccurate. We refer to this variation of the query as "local IN". To correct how the query works when data is spread randomly across the cluster servers, you could specify distributed\_table inside a subquery. The query would look like this: ``` SELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34) ``` This query will be sent to all remote servers as ``` SELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34) ``` The subquery will begin running on each remote server. Since the subquery uses a distributed table, the subquery that is on each remote server will be resent to every remote server as ``` SELECT UserID FROM local_table WHERE CounterID = 34 ``` For example, if you have a cluster of 100 servers, executing the entire query will require 10,000 elementary requests, which is generally considered unacceptable. In such cases, you should always use GLOBAL IN instead of IN. Let's look at how it works for the query ``` SELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID GLOBAL IN (SELECT UserID FROM distributed_table WHERE CounterID = 34) ``` The requestor server will run the subquery ``` SELECT UserID FROM distributed_table WHERE CounterID = 34 ``` and the result will be put in a temporary table in RAM. Then the request will be sent to each remote server as ``` SELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID GLOBAL IN _data1 ``` and the temporary table `_data1` will be sent to every remote server with the query (the name of the temporary table is implementation-defined). This is more optimal than using the normal IN. However, keep the following points in mind: 1. When creating a temporary table, data is not made unique. To reduce the volume of data transmitted over the network, specify DISTINCT in the subquery. (You don't need to do this for a normal IN.) 2. The temporary table will be sent to all the remote servers. Transmission does not account for network topology. For example, if 10 remote servers reside in a datacenter that is very remote in relation to the requestor server, the data will be sent 10 times over the channel to the remote datacenter. Try to avoid large data sets when using GLOBAL IN. 3. When transmitting data to remote servers, restrictions on network bandwidth are not configurable. You might overload the network. 4. Try to distribute data across servers so that you don't need to use GLOBAL IN on a regular basis. 5. If you need to use GLOBAL IN often, plan the location of the ClickHouse cluster so that a single group of replicas resides in no more than one data center with a fast network between them, so that a query can be processed entirely within a single data center. It also makes sense to specify a local table in the `GLOBAL IN` clause, in case this local table is only available on the requestor server and you want to use data from it on remote servers. ### Extreme Values[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#extreme-values "Permanent link") In addition to results, you can also get minimum and maximum values for the results columns. To do this, set the extremes setting to 1. Minimums and maximums are calculated for numeric types, dates, and dates with times. For other columns, the default values are output. An extra two rows are calculated – the minimums and maximums, respectively. These extra two rows are output in JSON\, TabSeparated\, and Pretty\* formats, separate from the other rows. They are not output for other formats. In JSON\* formats, the extreme values are output in a separate 'extremes' field. In TabSeparated\* formats, the row comes after the main result, and after 'totals' if present. It is preceded by an empty row (after the other data). In Pretty\* formats, the row is output as a separate table after the main result, and after 'totals' if present. Extreme values are calculated for rows that have passed through LIMIT. However, when using 'LIMIT offset, size', the rows before 'offset' are included in 'extremes'. In stream requests, the result may also include a small number of rows that passed through LIMIT. ### Notes[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#notes "Permanent link") The `GROUP BY` and `ORDER BY` clauses do not support positional arguments. This contradicts MySQL, but conforms to standard SQL. For example, `GROUP BY 1, 2` will be interpreted as grouping by constants (i.e. aggregation of all rows into one). You can use synonyms (`AS` aliases) in any part of a query. You can put an asterisk in any part of a query instead of an expression. When the query is analyzed, the asterisk is expanded to a list of all table columns (excluding the `MATERIALIZED` and `ALIAS` columns). There are only a few cases when using an asterisk is justified: - When creating a table dump. - For tables containing just a few columns, such as system tables. - For getting information about what columns are in a table. In this case, set `LIMIT 1`. But it is better to use the `DESC TABLE` query. - When there is strong filtration on a small number of columns using `PREWHERE`. - In subqueries (since columns that aren't needed for the external query are excluded from subqueries). In all other cases, we don't recommend using the asterisk, since it only gives you the drawbacks of a columnar DBMS instead of the advantages. In other words using the asterisk is not recommended. [Original article](https://clickhouse.yandex/docs/en/query_language/select/) [Previous Buffer](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/buffer/ "Buffer") [Next INSERT INTO](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/insert_into/ "INSERT INTO") ©2016–2019 Yandex LLC
Readable Markdown	## SELECT Queries Syntax[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-queries-syntax "Permanent link") `SELECT` performs data retrieval. ``` SELECT [DISTINCT] expr_list [FROM [db.]table \| (subquery) \| table_function] [FINAL] [SAMPLE sample_coeff] [ARRAY JOIN ...] [GLOBAL] [ANY\|ALL] [INNER\|LEFT\|RIGHT\|FULL\|CROSS] [OUTER] JOIN (subquery)\|table USING columns_list [PREWHERE expr] [WHERE expr] [GROUP BY expr_list] [WITH TOTALS] [HAVING expr] [ORDER BY expr_list] [LIMIT [n, ]m] [UNION ALL ...] [INTO OUTFILE filename] [FORMAT format] [LIMIT n BY columns] ``` All the clauses are optional, except for the required list of expressions immediately after SELECT. The clauses below are described in almost the same order as in the query execution conveyor. If the query omits the `DISTINCT`, `GROUP BY` and `ORDER BY` clauses and the `IN` and `JOIN` subqueries, the query will be completely stream processed, using O(1) amount of RAM. Otherwise, the query might consume a lot of RAM if the appropriate restrictions are not specified: `max_memory_usage`, `max_rows_to_group_by`, `max_rows_to_sort`, `max_rows_in_distinct`, `max_bytes_in_distinct`, `max_rows_in_set`, `max_bytes_in_set`, `max_rows_in_join`, `max_bytes_in_join`, `max_bytes_before_external_sort`, `max_bytes_before_external_group_by`. For more information, see the section "Settings". It is possible to use external sorting (saving temporary tables to a disk) and external aggregation. `The system does not have "merge join"`. ### FROM Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#from-clause "Permanent link") If the FROM clause is omitted, data will be read from the `system.one` table. The 'system.one' table contains exactly one row (this table fulfills the same purpose as the DUAL table found in other DBMSs). The FROM clause specifies the table to read data from, or a subquery, or a table function; ARRAY JOIN and the regular JOIN may also be included (see below). Instead of a table, the SELECT subquery may be specified in brackets. In this case, the subquery processing pipeline will be built into the processing pipeline of an external query. In contrast to standard SQL, a synonym does not need to be specified after a subquery. For compatibility, it is possible to write 'AS name' after a subquery, but the specified name isn't used anywhere. A table function may be specified instead of a table. For more information, see the section "Table functions". To execute a query, all the columns listed in the query are extracted from the appropriate table. Any columns not needed for the external query are thrown out of the subqueries. If a query does not list any columns (for example, SELECT count() FROM t), some column is extracted from the table anyway (the smallest one is preferred), in order to calculate the number of rows. The FINAL modifier can be used only for a SELECT from a CollapsingMergeTree table. When you specify FINAL, data is selected fully "collapsed". Keep in mind that using FINAL leads to a selection that includes columns related to the primary key, in addition to the columns specified in the SELECT. Additionally, the query will be executed in a single stream, and data will be merged during query execution. This means that when using FINAL, the query is processed more slowly. In most cases, you should avoid using FINAL. For more information, see the section "CollapsingMergeTree engine". ### SAMPLE Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-clause "Permanent link") The `SAMPLE` clause allows for approximated query processing. Approximated query processing is only supported by the tables in the `MergeTree` family, and only if the sampling expression was specified during table creation (see [MergeTree engine](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/mergetree/)). The features of data sampling are listed below: - Data sampling is a deterministic mechanism. The result of the same `SELECT .. SAMPLE` query is always the same. - Sampling works consistently for different tables. For tables with a single sampling key, a sample with the same coefficient always selects the same subset of possible data. For example, a sample of user IDs takes rows with the same subset of all the possible user IDs from different tables. This allows using the sample in subqueries in the `IN` clause, as well as for manually correlating results of different queries with samples. - Sampling allows reading less data from a disk. Note that for this you must specify the sampling key correctly. For more details see [Creating a MergeTree Table](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/mergetree/#table_engine-mergetree-creating-a-table). The `SAMPLE` clause can be specified in several ways: - `SAMPLE k`, where `k` is a decimal number from 0 to 1. The query is executed on `k` percent of data. For example, `SAMPLE 0.1` runs the query on 10% of data. [Read more](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-k) - `SAMPLE n`, where `n` is a sufficiently large integer. The query is executed on a sample of at least `n` rows (but not significantly more than this). For example, `SAMPLE 10000000` runs the query on a minimum of 10,000,000 rows. [Read more](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-n) - `SAMPLE k OFFSET m` where `k` and `m` are numbers from 0 to 1. The query is executed on a sample of `k` percent of the data. The data used for the sample is offset by `m` percent. [Read more](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-offset) #### SAMPLE k[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-k "Permanent link") In a `SAMPLE k` clause, `k` is a percent amount of data that the sample is taken from. The example is shown below: ``` SELECT Title, count() * 10 AS PageViews FROM hits_distributed SAMPLE 0.1 WHERE CounterID = 34 GROUP BY Title ORDER BY PageViews DESC LIMIT 1000 ``` In this example, the query is executed on a sample from 0.1 (10%) of data. Values of aggregate functions are not corrected automatically, so to get an approximate result, the value 'count()' is manually multiplied by 10. #### SAMPLE n[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-n "Permanent link") In this case, the query is executed on a sample of at least `n` rows, where `n` is a sufficiently large integer. For example, `SAMPLE 10000000`. Since the minimum unit for data reading is one granule (its size is set by the `index_granularity` setting), it makes sense to set a sample that is much larger than the size of the granule. When using the `SAMPLE n` clause, the relative coefficient is calculated dynamically. Since you do not know which relative percent of data was processed, you do not know the coefficient the aggregate functions should be multiplied by (for example, you do not know if the `SAMPLE 1000000` was taken from a set of 10,000,000 rows or from a set of 1,000,000,000 rows). In this case, use the `_sample_factor` column to get the approximate result. The `_sample_factor` is a virtual column that ClickHouse stores relative coefficients in. This column is created automatically when you create a table with the specified sampling key. The usage example is shown below: ``` SELECT sum(Duration * _sample_factor) FROM visits SAMPLE 10000000 ``` If you need to get the approximate count of rows in a `SELECT .. SAMPLE n` query, get the sum() of `_sample_factor` column instead of counting `count(column * _sample_factor)` value. For example: ``` SELECT sum(_sample_factor) FROM visits SAMPLE 10000000 ``` Note that to calculate the average in a `SELECT .. SAMPLE n` query, you do not need to use `_sample_factor` column: ``` SELECT avg(Duration) FROM visits SAMPLE 10000000 ``` #### SAMPLE k OFFSET m[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-sample-offset "Permanent link") You can specify the `SAMPLE k OFFSET m` clause, where `k` and `m` are numbers from 0 to 1. Examples are shown below. Example 1. ``` SAMPLE 1/10 ``` In this example, the sample is the 1/10th of all data: `[++------------------]` Example 2. ``` SAMPLE 1/10 OFFSET 1/2 ``` Here, the sample of 10% is taken from the second half of data. `[----------++--------]` ### ARRAY JOIN Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-array-join-clause "Permanent link") Allows executing JOIN with an array or nested data structure. The intent is similar to the 'arrayJoin' function, but its functionality is broader. `ARRAY JOIN` is essentially `INNER JOIN` with an array. Example: ``` :) CREATE TABLE arrays_test (s String, arr Array(UInt8)) ENGINE = Memory CREATE TABLE arrays_test ( s String, arr Array(UInt8) ) ENGINE = Memory Ok. 0 rows in set. Elapsed: 0.001 sec. :) INSERT INTO arrays_test VALUES ('Hello', [1,2]), ('World', [3,4,5]), ('Goodbye', []) INSERT INTO arrays_test VALUES Ok. 3 rows in set. Elapsed: 0.001 sec. :) SELECT * FROM arrays_test SELECT * FROM arrays_test ┌─s───────┬─arr─────┐ │ Hello │ [1,2] │ │ World │ [3,4,5] │ │ Goodbye │ [] │ └─────────┴─────────┘ 3 rows in set. Elapsed: 0.001 sec. :) SELECT s, arr FROM arrays_test ARRAY JOIN arr SELECT s, arr FROM arrays_test ARRAY JOIN arr ┌─s─────┬─arr─┐ │ Hello │ 1 │ │ Hello │ 2 │ │ World │ 3 │ │ World │ 4 │ │ World │ 5 │ └───────┴─────┘ 5 rows in set. Elapsed: 0.001 sec. ``` An alias can be specified for an array in the ARRAY JOIN clause. In this case, an array item can be accessed by this alias, but the array itself by the original name. Example: ``` :) SELECT s, arr, a FROM arrays_test ARRAY JOIN arr AS a SELECT s, arr, a FROM arrays_test ARRAY JOIN arr AS a ┌─s─────┬─arr─────┬─a─┐ │ Hello │ [1,2] │ 1 │ │ Hello │ [1,2] │ 2 │ │ World │ [3,4,5] │ 3 │ │ World │ [3,4,5] │ 4 │ │ World │ [3,4,5] │ 5 │ └───────┴─────────┴───┘ 5 rows in set. Elapsed: 0.001 sec. ``` Multiple arrays of the same size can be comma-separated in the ARRAY JOIN clause. In this case, JOIN is performed with them simultaneously (the direct sum, not the direct product). Example: ``` :) SELECT s, arr, a, num, mapped FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(x -> x + 1, arr) AS mapped SELECT s, arr, a, num, mapped FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num, arrayMap(lambda(tuple(x), plus(x, 1)), arr) AS mapped ┌─s─────┬─arr─────┬─a─┬─num─┬─mapped─┐ │ Hello │ [1,2] │ 1 │ 1 │ 2 │ │ Hello │ [1,2] │ 2 │ 2 │ 3 │ │ World │ [3,4,5] │ 3 │ 1 │ 4 │ │ World │ [3,4,5] │ 4 │ 2 │ 5 │ │ World │ [3,4,5] │ 5 │ 3 │ 6 │ └───────┴─────────┴───┴─────┴────────┘ 5 rows in set. Elapsed: 0.002 sec. :) SELECT s, arr, a, num, arrayEnumerate(arr) FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num SELECT s, arr, a, num, arrayEnumerate(arr) FROM arrays_test ARRAY JOIN arr AS a, arrayEnumerate(arr) AS num ┌─s─────┬─arr─────┬─a─┬─num─┬─arrayEnumerate(arr)─┐ │ Hello │ [1,2] │ 1 │ 1 │ [1,2] │ │ Hello │ [1,2] │ 2 │ 2 │ [1,2] │ │ World │ [3,4,5] │ 3 │ 1 │ [1,2,3] │ │ World │ [3,4,5] │ 4 │ 2 │ [1,2,3] │ │ World │ [3,4,5] │ 5 │ 3 │ [1,2,3] │ └───────┴─────────┴───┴─────┴─────────────────────┘ 5 rows in set. Elapsed: 0.002 sec. ``` ARRAY JOIN also works with nested data structures. Example: ``` :) CREATE TABLE nested_test (s String, nest Nested(x UInt8, y UInt32)) ENGINE = Memory CREATE TABLE nested_test ( s String, nest Nested( x UInt8, y UInt32) ) ENGINE = Memory Ok. 0 rows in set. Elapsed: 0.006 sec. :) INSERT INTO nested_test VALUES ('Hello', [1,2], [10,20]), ('World', [3,4,5], [30,40,50]), ('Goodbye', [], []) INSERT INTO nested_test VALUES Ok. 3 rows in set. Elapsed: 0.001 sec. :) SELECT * FROM nested_test SELECT * FROM nested_test ┌─s───────┬─nest.x──┬─nest.y─────┐ │ Hello │ [1,2] │ [10,20] │ │ World │ [3,4,5] │ [30,40,50] │ │ Goodbye │ [] │ [] │ └─────────┴─────────┴────────────┘ 3 rows in set. Elapsed: 0.001 sec. :) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest SELECT s, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN nest ┌─s─────┬─nest.x─┬─nest.y─┐ │ Hello │ 1 │ 10 │ │ Hello │ 2 │ 20 │ │ World │ 3 │ 30 │ │ World │ 4 │ 40 │ │ World │ 5 │ 50 │ └───────┴────────┴────────┘ 5 rows in set. Elapsed: 0.001 sec. ``` When specifying names of nested data structures in ARRAY JOIN, the meaning is the same as ARRAY JOIN with all the array elements that it consists of. Example: ``` :) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x, nest.y SELECT s, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN `nest.x`, `nest.y` ┌─s─────┬─nest.x─┬─nest.y─┐ │ Hello │ 1 │ 10 │ │ Hello │ 2 │ 20 │ │ World │ 3 │ 30 │ │ World │ 4 │ 40 │ │ World │ 5 │ 50 │ └───────┴────────┴────────┘ 5 rows in set. Elapsed: 0.001 sec. ``` This variation also makes sense: ``` :) SELECT s, nest.x, nest.y FROM nested_test ARRAY JOIN nest.x SELECT s, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN `nest.x` ┌─s─────┬─nest.x─┬─nest.y─────┐ │ Hello │ 1 │ [10,20] │ │ Hello │ 2 │ [10,20] │ │ World │ 3 │ [30,40,50] │ │ World │ 4 │ [30,40,50] │ │ World │ 5 │ [30,40,50] │ └───────┴────────┴────────────┘ 5 rows in set. Elapsed: 0.001 sec. ``` An alias may be used for a nested data structure, in order to select either the JOIN result or the source array. Example: ``` :) SELECT s, n.x, n.y, nest.x, nest.y FROM nested_test ARRAY JOIN nest AS n SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y` FROM nested_test ARRAY JOIN nest AS n ┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┐ │ Hello │ 1 │ 10 │ [1,2] │ [10,20] │ │ Hello │ 2 │ 20 │ [1,2] │ [10,20] │ │ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │ │ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │ │ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │ └───────┴─────┴─────┴─────────┴────────────┘ 5 rows in set. Elapsed: 0.001 sec. ``` Example of using the arrayEnumerate function: ``` :) SELECT s, n.x, n.y, nest.x, nest.y, num FROM nested_test ARRAY JOIN nest AS n, arrayEnumerate(nest.x) AS num SELECT s, `n.x`, `n.y`, `nest.x`, `nest.y`, num FROM nested_test ARRAY JOIN nest AS n, arrayEnumerate(`nest.x`) AS num ┌─s─────┬─n.x─┬─n.y─┬─nest.x──┬─nest.y─────┬─num─┐ │ Hello │ 1 │ 10 │ [1,2] │ [10,20] │ 1 │ │ Hello │ 2 │ 20 │ [1,2] │ [10,20] │ 2 │ │ World │ 3 │ 30 │ [3,4,5] │ [30,40,50] │ 1 │ │ World │ 4 │ 40 │ [3,4,5] │ [30,40,50] │ 2 │ │ World │ 5 │ 50 │ [3,4,5] │ [30,40,50] │ 3 │ └───────┴─────┴─────┴─────────┴────────────┴─────┘ 5 rows in set. Elapsed: 0.002 sec. ``` The query can only specify a single ARRAY JOIN clause. The corresponding conversion can be performed before the WHERE/PREWHERE clause (if its result is needed in this clause), or after completing WHERE/PREWHERE (to reduce the volume of calculations). ### JOIN Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-join "Permanent link") Joins the data in the normal [SQL JOIN](https://en.wikipedia.org/wiki/Join_\(SQL\)) sense. Note Not related to [ARRAY JOIN](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-array-join-clause). ``` SELECT <expr_list> FROM <left_subquery> [GLOBAL] [ANY\|ALL] [INNER\|LEFT\|RIGHT\|FULL\|CROSS] [OUTER] JOIN <right_subquery> (ON <expr_list>)\|(USING <column_list>) ... ``` The table names can be specified instead of `<left_subquery>` and `<right_subquery>`. This is equivalent to the `SELECT * FROM table` subquery, except in a special case when the table has the [Join](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/join/) engine – an array prepared for joining. Supported types of `JOIN` - `INNER JOIN` (or `JOIN`) - `LEFT JOIN` (or `LEFT OUTER JOIN`) - `RIGHT JOIN` (or `RIGHT OUTER JOIN`) - `FULL JOIN` (or `FULL OUTER JOIN`) - `CROSS JOIN` (or `,` ) See standard [SQL JOIN](https://en.wikipedia.org/wiki/Join_\(SQL\)) description. ANY or ALL strictness If `ALL` is specified and the right table has several matching rows, the data will be multiplied by the number of these rows. This is the normal `JOIN` behavior for standard SQL. If `ANY` is specified and the right table has several matching rows, only the first one found is joined. If the right table has only one matching row, the results of `ANY` and `ALL` are the same. To set the default strictness value, use the session configuration parameter [join\_default\_strictness](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/settings/#settings-join_default_strictness). GLOBAL JOIN When using a normal `JOIN`, the query is sent to remote servers. Subqueries are run on each of them in order to make the right table, and the join is performed with this table. In other words, the right table is formed on each server separately. When using `GLOBAL ... JOIN`, first the requestor server runs a subquery to calculate the right table. This temporary table is passed to each remote server, and queries are run on them using the temporary data that was transmitted. Be careful when using `GLOBAL`. For more information, see the section [Distributed subqueries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distributed-subqueries). Usage Recommendations All columns that are not needed for the `JOIN` are deleted from the subquery. When running a `JOIN`, there is no optimization of the order of execution in relation to other stages of the query. The join (a search in the right table) is run before filtering in `WHERE` and before aggregation. In order to explicitly set the processing order, we recommend running a `JOIN` subquery with a subquery. Example: ``` SELECT CounterID, hits, visits FROM ( SELECT CounterID, count() AS hits FROM test.hits GROUP BY CounterID ) ANY LEFT JOIN ( SELECT CounterID, sum(Sign) AS visits FROM test.visits GROUP BY CounterID ) USING CounterID ORDER BY hits DESC LIMIT 10 ``` ``` ┌─CounterID─┬───hits─┬─visits─┐ │ 1143050 │ 523264 │ 13665 │ │ 731962 │ 475698 │ 102716 │ │ 722545 │ 337212 │ 108187 │ │ 722889 │ 252197 │ 10547 │ │ 2237260 │ 196036 │ 9522 │ │ 23057320 │ 147211 │ 7689 │ │ 722818 │ 90109 │ 17847 │ │ 48221 │ 85379 │ 4652 │ │ 19762435 │ 77807 │ 7026 │ │ 722884 │ 77492 │ 11056 │ └───────────┴────────┴────────┘ ``` Subqueries don't allow you to set names or use them for referencing a column from a specific subquery. The columns specified in `USING` must have the same names in both subqueries, and the other columns must be named differently. You can use aliases to change the names of columns in subqueries (the example uses the aliases 'hits' and 'visits'). The `USING` clause specifies one or more columns to join, which establishes the equality of these columns. The list of columns is set without brackets. More complex join conditions are not supported. The right table (the subquery result) resides in RAM. If there isn't enough memory, you can't run a `JOIN`. Each time a query is run with the same `JOIN`, the subquery is run again because the result is not cached. To avoid this, use the special [Join](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/table_engines/join/) table engine, which is a prepared array for joining that is always in RAM. In some cases, it is more efficient to use `IN` instead of `JOIN`. Among the various types of `JOIN`, the most efficient is `ANY LEFT JOIN`, then `ANY INNER JOIN`. The least efficient are `ALL LEFT JOIN` and `ALL INNER JOIN`. If you need a `JOIN` for joining with dimension tables (these are relatively small tables that contain dimension properties, such as names for advertising campaigns), a `JOIN` might not be very convenient due to the bulky syntax and the fact that the right table is re-accessed for every query. For such cases, there is an "external dictionaries" feature that you should use instead of `JOIN`. For more information, see the section [External dictionaries](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/dicts/external_dicts/). #### Processing of Empty or NULL Cells[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#processing-of-empty-or-null-cells "Permanent link") While joining tables, the empty cells may appear. The setting [join\_use\_nulls](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/settings/#settings-join_use_nulls) define how ClickHouse fills these cells. If the `JOIN` keys are [Nullable](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/data_types/nullable/) fields, the rows where at least one of the keys has the value [NULL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/syntax/#null-literal) are not joined. ### WHERE Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#where-clause "Permanent link") If there is a WHERE clause, it must contain an expression with the UInt8 type. This is usually an expression with comparison and logical operators. This expression will be used for filtering data before all other transformations. If indexes are supported by the database table engine, the expression is evaluated on the ability to use indexes. ### PREWHERE Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#prewhere-clause "Permanent link") This clause has the same meaning as the WHERE clause. The difference is in which data is read from the table. When using PREWHERE, first only the columns necessary for executing PREWHERE are read. Then the other columns are read that are needed for running the query, but only those blocks where the PREWHERE expression is true. It makes sense to use PREWHERE if there are filtration conditions that are used by a minority of the columns in the query, but that provide strong data filtration. This reduces the volume of data to read. For example, it is useful to write PREWHERE for queries that extract a large number of columns, but that only have filtration for a few columns. PREWHERE is only supported by tables from the `MergeTree` family. A query may simultaneously specify PREWHERE and WHERE. In this case, PREWHERE precedes WHERE. If the 'optimize\_move\_to\_prewhere' setting is set to 1 and PREWHERE is omitted, the system uses heuristics to automatically move parts of expressions from WHERE to PREWHERE. ### GROUP BY Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-group-by-clause "Permanent link") This is one of the most important parts of a column-oriented DBMS. If there is a GROUP BY clause, it must contain a list of expressions. Each expression will be referred to here as a "key". All the expressions in the SELECT, HAVING, and ORDER BY clauses must be calculated from keys or from aggregate functions. In other words, each column selected from the table must be used either in keys or inside aggregate functions. If a query contains only table columns inside aggregate functions, the GROUP BY clause can be omitted, and aggregation by an empty set of keys is assumed. Example: ``` SELECT count(), median(FetchTiming > 60 ? 60 : FetchTiming), count() - sum(Refresh) FROM hits ``` However, in contrast to standard SQL, if the table doesn't have any rows (either there aren't any at all, or there aren't any after using WHERE to filter), an empty result is returned, and not the result from one of the rows containing the initial values of aggregate functions. As opposed to MySQL (and conforming to standard SQL), you can't get some value of some column that is not in a key or aggregate function (except constant expressions). To work around this, you can use the 'any' aggregate function (get the first encountered value) or 'min/max'. Example: ``` SELECT domainWithoutWWW(URL) AS domain, count(), any(Title) AS title -- getting the first occurred page header for each domain. FROM hits GROUP BY domain ``` For every different key value encountered, GROUP BY calculates a set of aggregate function values. GROUP BY is not supported for array columns. A constant can't be specified as arguments for aggregate functions. Example: sum(1). Instead of this, you can get rid of the constant. Example: `count()`. #### NULL processing[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#null-processing "Permanent link") For grouping, ClickHouse interprets [NULL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/syntax/) as a value, and `NULL=NULL`. Here's an example to show what this means. Assume you have this table: ``` ┌─x─┬────y─┐ │ 1 │ 2 │ │ 2 │ ᴺᵁᴸᴸ │ │ 3 │ 2 │ │ 3 │ 3 │ │ 3 │ ᴺᵁᴸᴸ │ └───┴──────┘ ``` The query `SELECT sum(x), y FROM t_null_big GROUP BY y` results in: ``` ┌─sum(x)─┬────y─┐ │ 4 │ 2 │ │ 3 │ 3 │ │ 5 │ ᴺᵁᴸᴸ │ └────────┴──────┘ ``` You can see that `GROUP BY` for `У = NULL` summed up `x`, as if `NULL` is this value. If you pass several keys to `GROUP BY`, the result will give you all the combinations of the selection, as if `NULL` were a specific value. #### WITH TOTALS Modifier[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#with-totals-modifier "Permanent link") If the WITH TOTALS modifier is specified, another row will be calculated. This row will have key columns containing default values (zeros or empty lines), and columns of aggregate functions with the values calculated across all the rows (the "total" values). This extra row is output in JSON\, TabSeparated\, and Pretty\ formats, separately from the other rows. In the other formats, this row is not output. In JSON\* formats, this row is output as a separate 'totals' field. In TabSeparated\* formats, the row comes after the main result, preceded by an empty row (after the other data). In Pretty\* formats, the row is output as a separate table after the main result. `WITH TOTALS` can be run in different ways when HAVING is present. The behavior depends on the 'totals\_mode' setting. By default, `totals_mode = 'before_having'`. In this case, 'totals' is calculated across all rows, including the ones that don't pass through HAVING and 'max\_rows\_to\_group\_by'. The other alternatives include only the rows that pass through HAVING in 'totals', and behave differently with the setting `max_rows_to_group_by` and `group_by_overflow_mode = 'any'`. `after_having_exclusive` – Don't include rows that didn't pass through `max_rows_to_group_by`. In other words, 'totals' will have less than or the same number of rows as it would if `max_rows_to_group_by` were omitted. `after_having_inclusive` – Include all the rows that didn't pass through 'max\_rows\_to\_group\_by' in 'totals'. In other words, 'totals' will have more than or the same number of rows as it would if `max_rows_to_group_by` were omitted. `after_having_auto` – Count the number of rows that passed through HAVING. If it is more than a certain amount (by default, 50%), include all the rows that didn't pass through 'max\_rows\_to\_group\_by' in 'totals'. Otherwise, do not include them. `totals_auto_threshold` – By default, 0.5. The coefficient for `after_having_auto`. If `max_rows_to_group_by` and `group_by_overflow_mode = 'any'` are not used, all variations of `after_having` are the same, and you can use any of them (for example, `after_having_auto`). You can use WITH TOTALS in subqueries, including subqueries in the JOIN clause (in this case, the respective total values are combined). #### GROUP BY in External Memory[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-group-by-in-external-memory "Permanent link") You can enable dumping temporary data to the disk to restrict memory usage during GROUP BY. The `max_bytes_before_external_group_by` setting determines the threshold RAM consumption for dumping GROUP BY temporary data to the file system. If set to 0 (the default), it is disabled. When using `max_bytes_before_external_group_by`, we recommend that you set max\_memory\_usage about twice as high. This is necessary because there are two stages to aggregation: reading the date and forming intermediate data (1) and merging the intermediate data (2). Dumping data to the file system can only occur during stage 1. If the temporary data wasn't dumped, then stage 2 might require up to the same amount of memory as in stage 1. For example, if `max_memory_usage` was set to 10000000000 and you want to use external aggregation, it makes sense to set `max_bytes_before_external_group_by` to 10000000000, and max\_memory\_usage to 20000000000. When external aggregation is triggered (if there was at least one dump of temporary data), maximum consumption of RAM is only slightly more than `max_bytes_before_external_group_by`. With distributed query processing, external aggregation is performed on remote servers. In order for the requestor server to use only a small amount of RAM, set `distributed_aggregation_memory_efficient` to 1. When merging data flushed to the disk, as well as when merging results from remote servers when the `distributed_aggregation_memory_efficient` setting is enabled, consumes up to 1/256 \* the number of threads from the total amount of RAM. When external aggregation is enabled, if there was less than `max_bytes_before_external_group_by` of data (i.e. data was not flushed), the query runs just as fast as without external aggregation. If any temporary data was flushed, the run time will be several times longer (approximately three times). If you have an ORDER BY with a small LIMIT after GROUP BY, then the ORDER BY CLAUSE will not use significant amounts of RAM. But if the ORDER BY doesn't have LIMIT, don't forget to enable external sorting (`max_bytes_before_external_sort`). ### LIMIT N BY Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#limit-n-by-clause "Permanent link") LIMIT N BY COLUMNS selects the top N rows for each group of COLUMNS. LIMIT N BY is not related to LIMIT; they can both be used in the same query. The key for LIMIT N BY can contain any number of columns or expressions. Example: ``` SELECT domainWithoutWWW(URL) AS domain, domainWithoutWWW(REFERRER_URL) AS referrer, device_type, count() cnt FROM hits GROUP BY domain, referrer, device_type ORDER BY cnt DESC LIMIT 5 BY domain, device_type LIMIT 100 ``` The query will select the top 5 referrers for each `domain, device_type` pair, but not more than 100 rows (`LIMIT n BY + LIMIT`). ### HAVING Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#having-clause "Permanent link") Allows filtering the result received after GROUP BY, similar to the WHERE clause. WHERE and HAVING differ in that WHERE is performed before aggregation (GROUP BY), while HAVING is performed after it. If aggregation is not performed, HAVING can't be used. ### ORDER BY Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#order-by-clause "Permanent link") The ORDER BY clause contains a list of expressions, which can each be assigned DESC or ASC (the sorting direction). If the direction is not specified, ASC is assumed. ASC is sorted in ascending order, and DESC in descending order. The sorting direction applies to a single expression, not to the entire list. Example: `ORDER BY Visits DESC, SearchPhrase` For sorting by String values, you can specify collation (comparison). Example: `ORDER BY SearchPhrase COLLATE 'tr'` - for sorting by keyword in ascending order, using the Turkish alphabet, case insensitive, assuming that strings are UTF-8 encoded. COLLATE can be specified or not for each expression in ORDER BY independently. If ASC or DESC is specified, COLLATE is specified after it. When using COLLATE, sorting is always case-insensitive. We only recommend using COLLATE for final sorting of a small number of rows, since sorting with COLLATE is less efficient than normal sorting by bytes. Rows that have identical values for the list of sorting expressions are output in an arbitrary order, which can also be nondeterministic (different each time). If the ORDER BY clause is omitted, the order of the rows is also undefined, and may be nondeterministic as well. `NaN` and `NULL` sorting order: - With the modifier `NULLS FIRST` — First `NULL`, then `NaN`, then other values. - With the modifier `NULLS LAST` — First the values, then `NaN`, then `NULL`. - Default — The same as with the `NULLS LAST` modifier. Example: For the table ``` ┌─x─┬────y─┐ │ 1 │ ᴺᵁᴸᴸ │ │ 2 │ 2 │ │ 1 │ nan │ │ 2 │ 2 │ │ 3 │ 4 │ │ 5 │ 6 │ │ 6 │ nan │ │ 7 │ ᴺᵁᴸᴸ │ │ 6 │ 7 │ │ 8 │ 9 │ └───┴──────┘ ``` Run the query `SELECT * FROM t_null_nan ORDER BY y NULLS FIRST` to get: ``` ┌─x─┬────y─┐ │ 1 │ ᴺᵁᴸᴸ │ │ 7 │ ᴺᵁᴸᴸ │ │ 1 │ nan │ │ 6 │ nan │ │ 2 │ 2 │ │ 2 │ 2 │ │ 3 │ 4 │ │ 5 │ 6 │ │ 6 │ 7 │ │ 8 │ 9 │ └───┴──────┘ ``` When floating point numbers are sorted, NaNs are separate from the other values. Regardless of the sorting order, NaNs come at the end. In other words, for ascending sorting they are placed as if they are larger than all the other numbers, while for descending sorting they are placed as if they are smaller than the rest. Less RAM is used if a small enough LIMIT is specified in addition to ORDER BY. Otherwise, the amount of memory spent is proportional to the volume of data for sorting. For distributed query processing, if GROUP BY is omitted, sorting is partially done on remote servers, and the results are merged on the requestor server. This means that for distributed sorting, the volume of data to sort can be greater than the amount of memory on a single server. If there is not enough RAM, it is possible to perform sorting in external memory (creating temporary files on a disk). Use the setting `max_bytes_before_external_sort` for this purpose. If it is set to 0 (the default), external sorting is disabled. If it is enabled, when the volume of data to sort reaches the specified number of bytes, the collected data is sorted and dumped into a temporary file. After all data is read, all the sorted files are merged and the results are output. Files are written to the /var/lib/clickhouse/tmp/ directory in the config (by default, but you can use the 'tmp\_path' parameter to change this setting). Running a query may use more memory than 'max\_bytes\_before\_external\_sort'. For this reason, this setting must have a value significantly smaller than 'max\_memory\_usage'. As an example, if your server has 128 GB of RAM and you need to run a single query, set 'max\_memory\_usage' to 100 GB, and 'max\_bytes\_before\_external\_sort' to 80 GB. External sorting works much less effectively than sorting in RAM. ### SELECT Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-clause "Permanent link") The expressions specified in the SELECT clause are analyzed after the calculations for all the clauses listed above are completed. More specifically, expressions are analyzed that are above the aggregate functions, if there are any aggregate functions. The aggregate functions and everything below them are calculated during aggregation (GROUP BY). These expressions work as if they are applied to separate rows in the result. ### DISTINCT Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distinct "Permanent link") If DISTINCT is specified, only a single row will remain out of all the sets of fully matching rows in the result. The result will be the same as if GROUP BY were specified across all the fields specified in SELECT without aggregate functions. But there are several differences from GROUP BY: - DISTINCT can be applied together with GROUP BY. - When ORDER BY is omitted and LIMIT is defined, the query stops running immediately after the required number of different rows has been read. - Data blocks are output as they are processed, without waiting for the entire query to finish running. DISTINCT is not supported if SELECT has at least one array column. `DISTINCT` works with [NULL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/syntax/) as if `NULL` were a specific value, and `NULL=NULL`. In other words, in the `DISTINCT` results, different combinations with `NULL` only occur once. ### LIMIT Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#limit-clause "Permanent link") `LIMIT m` allows you to select the first `m` rows from the result. `LIMIT n`, m allows you to select the first `m` rows from the result after skipping the first `n` rows. `n` and `m` must be non-negative integers. If there isn't an ORDER BY clause that explicitly sorts results, the result may be arbitrary and nondeterministic. ### UNION ALL Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#union-all-clause "Permanent link") You can use UNION ALL to combine any number of queries. Example: ``` SELECT CounterID, 1 AS table, toInt64(count()) AS c FROM test.hits GROUP BY CounterID UNION ALL SELECT CounterID, 2 AS table, sum(Sign) AS c FROM test.visits GROUP BY CounterID HAVING c > 0 ``` Only UNION ALL is supported. The regular UNION (UNION DISTINCT) is not supported. If you need UNION DISTINCT, you can write SELECT DISTINCT from a subquery containing UNION ALL. Queries that are parts of UNION ALL can be run simultaneously, and their results can be mixed together. The structure of results (the number and type of columns) must match for the queries. But the column names can differ. In this case, the column names for the final result will be taken from the first query. Type casting is performed for unions. For example, if two queries being combined have the same field with non-`Nullable` and `Nullable` types from a compatible type, the resulting `UNION ALL` has a `Nullable` type field. Queries that are parts of UNION ALL can't be enclosed in brackets. ORDER BY and LIMIT are applied to separate queries, not to the final result. If you need to apply a conversion to the final result, you can put all the queries with UNION ALL in a subquery in the FROM clause. ### INTO OUTFILE Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#into-outfile-clause "Permanent link") Add the `INTO OUTFILE filename` clause (where filename is a string literal) to redirect query output to the specified file. In contrast to MySQL, the file is created on the client side. The query will fail if a file with the same filename already exists. This functionality is available in the command-line client and clickhouse-local (a query sent via HTTP interface will fail). The default output format is TabSeparated (the same as in the command-line client batch mode). ### FORMAT Clause[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#format-clause "Permanent link") Specify 'FORMAT format' to get data in any specified format. You can use this for convenience, or for creating dumps. For more information, see the section "Formats". If the FORMAT clause is omitted, the default format is used, which depends on both the settings and the interface used for accessing the DB. For the HTTP interface and the command-line client in batch mode, the default format is TabSeparated. For the command-line client in interactive mode, the default format is PrettyCompact (it has attractive and compact tables). When using the command-line client, data is passed to the client in an internal efficient format. The client independently interprets the FORMAT clause of the query and formats the data itself (thus relieving the network and the server from the load). ### IN Operators[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-in-operators "Permanent link") The `IN`, `NOT IN`, `GLOBAL IN`, and `GLOBAL NOT IN` operators are covered separately, since their functionality is quite rich. The left side of the operator is either a single column or a tuple. Examples: ``` SELECT UserID IN (123, 456) FROM ... SELECT (CounterID, UserID) IN ((34, 123), (101500, 456)) FROM ... ``` If the left side is a single column that is in the index, and the right side is a set of constants, the system uses the index for processing the query. Don't list too many values explicitly (i.e. millions). If a data set is large, put it in a temporary table (for example, see the section "External data for query processing"), then use a subquery. The right side of the operator can be a set of constant expressions, a set of tuples with constant expressions (shown in the examples above), or the name of a database table or SELECT subquery in brackets. If the right side of the operator is the name of a table (for example, `UserID IN users`), this is equivalent to the subquery `UserID IN (SELECT * FROM users)`. Use this when working with external data that is sent along with the query. For example, the query can be sent together with a set of user IDs loaded to the 'users' temporary table, which should be filtered. If the right side of the operator is a table name that has the Set engine (a prepared data set that is always in RAM), the data set will not be created over again for each query. The subquery may specify more than one column for filtering tuples. Example: ``` SELECT (CounterID, UserID) IN (SELECT CounterID, UserID FROM ...) FROM ... ``` The columns to the left and right of the IN operator should have the same type. The IN operator and subquery may occur in any part of the query, including in aggregate functions and lambda functions. Example: ``` SELECT EventDate, avg(UserID IN ( SELECT UserID FROM test.hits WHERE EventDate = toDate('2014-03-17') )) AS ratio FROM test.hits GROUP BY EventDate ORDER BY EventDate ASC ``` ``` ┌──EventDate─┬────ratio─┐ │ 2014-03-17 │ 1 │ │ 2014-03-18 │ 0.807696 │ │ 2014-03-19 │ 0.755406 │ │ 2014-03-20 │ 0.723218 │ │ 2014-03-21 │ 0.697021 │ │ 2014-03-22 │ 0.647851 │ │ 2014-03-23 │ 0.648416 │ └────────────┴──────────┘ ``` For each day after March 17th, count the percentage of pageviews made by users who visited the site on March 17th. A subquery in the IN clause is always run just one time on a single server. There are no dependent subqueries. #### NULL processing[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#null-processing_1 "Permanent link") During request processing, the IN operator assumes that the result of an operation with [NULL](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/syntax/) is always equal to `0`, regardless of whether `NULL` is on the right or left side of the operator. `NULL` values are not included in any dataset, do not correspond to each other and cannot be compared. Here is an example with the `t_null` table: ``` ┌─x─┬────y─┐ │ 1 │ ᴺᵁᴸᴸ │ │ 2 │ 3 │ └───┴──────┘ ``` Running the query `SELECT x FROM t_null WHERE y IN (NULL,3)` gives you the following result: ``` ┌─x─┐ │ 2 │ └───┘ ``` You can see that the row in which `y = NULL` is thrown out of the query results. This is because ClickHouse can't decide whether `NULL` is included in the `(NULL,3)` set, returns `0` as the result of the operation, and `SELECT` excludes this row from the final output. ``` SELECT y IN (NULL, 3) FROM t_null ┌─in(y, tuple(NULL, 3))─┐ │ 0 │ │ 1 │ └───────────────────────┘ ``` #### Distributed Subqueries[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#select-distributed-subqueries "Permanent link") There are two options for IN-s with subqueries (similar to JOINs): normal `IN` / `JOIN` and `GLOBAL IN` / `GLOBAL JOIN`. They differ in how they are run for distributed query processing. Attention Remember that the algorithms described below may work differently depending on the [settings](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/operations/settings/settings/) `distributed_product_mode` setting. When using the regular IN, the query is sent to remote servers, and each of them runs the subqueries in the `IN` or `JOIN` clause. When using `GLOBAL IN` / `GLOBAL JOINs`, first all the subqueries are run for `GLOBAL IN` / `GLOBAL JOINs`, and the results are collected in temporary tables. Then the temporary tables are sent to each remote server, where the queries are run using this temporary data. For a non-distributed query, use the regular `IN` / `JOIN`. Be careful when using subqueries in the `IN` / `JOIN` clauses for distributed query processing. Let's look at some examples. Assume that each server in the cluster has a normal local\_table. Each server also has a distributed\_table table with the Distributed type, which looks at all the servers in the cluster. For a query to the distributed\_table, the query will be sent to all the remote servers and run on them using the local\_table. For example, the query ``` SELECT uniq(UserID) FROM distributed_table ``` will be sent to all remote servers as ``` SELECT uniq(UserID) FROM local_table ``` and run on each of them in parallel, until it reaches the stage where intermediate results can be combined. Then the intermediate results will be returned to the requestor server and merged on it, and the final result will be sent to the client. Now let's examine a query with IN: ``` SELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34) ``` - Calculation of the intersection of audiences of two sites. This query will be sent to all remote servers as ``` SELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM local_table WHERE CounterID = 34) ``` In other words, the data set in the IN clause will be collected on each server independently, only across the data that is stored locally on each of the servers. This will work correctly and optimally if you are prepared for this case and have spread data across the cluster servers such that the data for a single UserID resides entirely on a single server. In this case, all the necessary data will be available locally on each server. Otherwise, the result will be inaccurate. We refer to this variation of the query as "local IN". To correct how the query works when data is spread randomly across the cluster servers, you could specify distributed\_table inside a subquery. The query would look like this: ``` SELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34) ``` This query will be sent to all remote servers as ``` SELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID IN (SELECT UserID FROM distributed_table WHERE CounterID = 34) ``` The subquery will begin running on each remote server. Since the subquery uses a distributed table, the subquery that is on each remote server will be resent to every remote server as ``` SELECT UserID FROM local_table WHERE CounterID = 34 ``` For example, if you have a cluster of 100 servers, executing the entire query will require 10,000 elementary requests, which is generally considered unacceptable. In such cases, you should always use GLOBAL IN instead of IN. Let's look at how it works for the query ``` SELECT uniq(UserID) FROM distributed_table WHERE CounterID = 101500 AND UserID GLOBAL IN (SELECT UserID FROM distributed_table WHERE CounterID = 34) ``` The requestor server will run the subquery ``` SELECT UserID FROM distributed_table WHERE CounterID = 34 ``` and the result will be put in a temporary table in RAM. Then the request will be sent to each remote server as ``` SELECT uniq(UserID) FROM local_table WHERE CounterID = 101500 AND UserID GLOBAL IN _data1 ``` and the temporary table `_data1` will be sent to every remote server with the query (the name of the temporary table is implementation-defined). This is more optimal than using the normal IN. However, keep the following points in mind: 1. When creating a temporary table, data is not made unique. To reduce the volume of data transmitted over the network, specify DISTINCT in the subquery. (You don't need to do this for a normal IN.) 2. The temporary table will be sent to all the remote servers. Transmission does not account for network topology. For example, if 10 remote servers reside in a datacenter that is very remote in relation to the requestor server, the data will be sent 10 times over the channel to the remote datacenter. Try to avoid large data sets when using GLOBAL IN. 3. When transmitting data to remote servers, restrictions on network bandwidth are not configurable. You might overload the network. 4. Try to distribute data across servers so that you don't need to use GLOBAL IN on a regular basis. 5. If you need to use GLOBAL IN often, plan the location of the ClickHouse cluster so that a single group of replicas resides in no more than one data center with a fast network between them, so that a query can be processed entirely within a single data center. It also makes sense to specify a local table in the `GLOBAL IN` clause, in case this local table is only available on the requestor server and you want to use data from it on remote servers. ### Extreme Values[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#extreme-values "Permanent link") In addition to results, you can also get minimum and maximum values for the results columns. To do this, set the extremes setting to 1. Minimums and maximums are calculated for numeric types, dates, and dates with times. For other columns, the default values are output. An extra two rows are calculated – the minimums and maximums, respectively. These extra two rows are output in JSON\, TabSeparated\, and Pretty\* formats, separate from the other rows. They are not output for other formats. In JSON\* formats, the extreme values are output in a separate 'extremes' field. In TabSeparated\* formats, the row comes after the main result, and after 'totals' if present. It is preceded by an empty row (after the other data). In Pretty\* formats, the row is output as a separate table after the main result, and after 'totals' if present. Extreme values are calculated for rows that have passed through LIMIT. However, when using 'LIMIT offset, size', the rows before 'offset' are included in 'extremes'. In stream requests, the result may also include a small number of rows that passed through LIMIT. ### Notes[¶](http://devdoc.net/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/#notes "Permanent link") The `GROUP BY` and `ORDER BY` clauses do not support positional arguments. This contradicts MySQL, but conforms to standard SQL. For example, `GROUP BY 1, 2` will be interpreted as grouping by constants (i.e. aggregation of all rows into one). You can use synonyms (`AS` aliases) in any part of a query. You can put an asterisk in any part of a query instead of an expression. When the query is analyzed, the asterisk is expanded to a list of all table columns (excluding the `MATERIALIZED` and `ALIAS` columns). There are only a few cases when using an asterisk is justified: - When creating a table dump. - For tables containing just a few columns, such as system tables. - For getting information about what columns are in a table. In this case, set `LIMIT 1`. But it is better to use the `DESC TABLE` query. - When there is strong filtration on a small number of columns using `PREWHERE`. - In subqueries (since columns that aren't needed for the external query are excluded from subqueries). In all other cases, we don't recommend using the asterisk, since it only gives you the drawbacks of a columnar DBMS instead of the advantages. In other words using the asterisk is not recommended. [Original article](https://clickhouse.yandex/docs/en/query_language/select/)
Shard	192 (laksa)
Root Hash	8504065000381170792
Unparsed URL	net,devdoc!/database/ClickhouseDocs_19.4.1.3-docs/query_language/select/ h80