🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 13 (from laksa198)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa013.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://note.nkmk.me/en/python-pandas-nan-dropna/\')), getAhrefsUnparsedNoserviceFromURL(\'https://note.nkmk.me/en/python-pandas-nan-dropna/\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/","crawl_time":1774614473,"first_indexed_time":1644736741,"http_code":200,"src_unparsed":"me,nkmk!note,\/en\/python-pandas-nan-dropna\/ s443","src_root_hash":"14415757146955323613","history_drop_reason":null,"meta_title":"pandas: Remove NaN (missing values) with dropna() | note.nkmk.me","meta_descriptions":["You can remove NaN from pandas.DataFrame and pandas.Series with the dropna() method. pandas.DataFrame.dropna — pandas 2.0.3 documentation pandas.Series.dropna — pandas 2.0.3 documentation Remove row ..."],"attrs_boilerpipe_text":"You can remove\nNaN\nfrom\npandas.DataFrame\nand\npandas.Series\nwith the\ndropna()\nmethod.\npandas.DataFrame.dropna — pandas 2.0.3 documentation\npandas.Series.dropna — pandas 2.0.3 documentation\nContents\nRemove rows\/columns where all elements are\nNaN\n:\nhow='all'\nRemove rows\/columns that contain at least one\nNaN\n:\nhow='any'\n(default)\nRemove rows\/columns according to the number of non-missing values:\nthresh\nRemove based on specific rows\/columns:\nsubset\nUpdate the original object:\ninplace\nFor\npandas.Series\nWhile this article primarily deals with\nNaN\n(Not a Number), it's important to note that in pandas,\nNone\nis also treated as a missing value.\nMissing values in pandas (nan, None, pd.NA)\nSee the following article on extracting, replacing, and counting missing values.\npandas: Find rows\/columns with NaN (missing values)\npandas: Replace NaN (missing values) with fillna()\npandas: Detect and count NaN (missing values) with isnull(), isna()\nThe sample code in this article uses pandas version\n2.0.3\n. As an example, read a CSV file with missing values.\nsample_pandas_normal_nan.csv\nimport\npandas\nas\npd\nprint\n(\npd\n.\n__version__\n)\n# 2.0.3\ndf\n=\npd\n.\nread_csv\n(\n'data\/src\/sample_pandas_normal_nan.csv'\n)\nprint\n(\ndf\n)\n#       name   age state  point  other\n# 0    Alice  24.0    NY    NaN    NaN\n# 1      NaN   NaN   NaN    NaN    NaN\n# 2  Charlie   NaN    CA    NaN    NaN\n# 3     Dave  68.0    TX   70.0    NaN\n# 4    Ellen   NaN    CA   88.0    NaN\n# 5    Frank  30.0   NaN    NaN    NaN\nRemove rows\/columns where all elements are\nNaN\n:\nhow='all'\nBy setting\nhow='all'\n, rows where all elements are\nNaN\nare removed.\nprint\n(\ndf\n.\ndropna\n(\nhow\n=\n'all'\n))\n#       name   age state  point  other\n# 0    Alice  24.0    NY    NaN    NaN\n# 2  Charlie   NaN    CA    NaN    NaN\n# 3     Dave  68.0    TX   70.0    NaN\n# 4    Ellen   NaN    CA   88.0    NaN\n# 5    Frank  30.0   NaN    NaN    NaN\nIf\naxis\nis set to\n1\nor\n'columns'\n, columns where all elements are\nNaN\nare removed.\nprint\n(\ndf\n.\ndropna\n(\nhow\n=\n'all'\n,\naxis\n=\n1\n))\n#       name   age state  point\n# 0    Alice  24.0    NY    NaN\n# 1      NaN   NaN   NaN    NaN\n# 2  Charlie   NaN    CA    NaN\n# 3     Dave  68.0    TX   70.0\n# 4    Ellen   NaN    CA   88.0\n# 5    Frank  30.0   NaN    NaN\nNote that if\naxis\nis set to\n0\nor\n'index'\n, rows are removed. Since the default value of\naxis\nis\n0\n, rows are removed if omitted, as shown in the first example.\nIn former versions, both rows and columns were removed with\naxis=[0, 1]\n, but since version\n1.0.0\n,\naxis\ncan no longer be specified with a list or tuple. If you want to remove both rows and columns, you can repeatedly apply\ndropna()\n.\n# print(df.dropna(how='all', axis=[0, 1]))\n# TypeError: supplying multiple axes to axis is no longer supported.\nprint\n(\ndf\n.\ndropna\n(\nhow\n=\n'all'\n)\n.\ndropna\n(\nhow\n=\n'all'\n,\naxis\n=\n1\n))\n#       name   age state  point\n# 0    Alice  24.0    NY    NaN\n# 2  Charlie   NaN    CA    NaN\n# 3     Dave  68.0    TX   70.0\n# 4    Ellen   NaN    CA   88.0\n# 5    Frank  30.0   NaN    NaN\nRemove rows\/columns that contain at least one\nNaN\n:\nhow='any'\n(default)\nTo use as an example, remove rows and columns where all values are\nNaN\n.\ndf2\n=\ndf\n.\ndropna\n(\nhow\n=\n'all'\n)\n.\ndropna\n(\nhow\n=\n'all'\n,\naxis\n=\n1\n)\nprint\n(\ndf2\n)\n#       name   age state  point\n# 0    Alice  24.0    NY    NaN\n# 2  Charlie   NaN    CA    NaN\n# 3     Dave  68.0    TX   70.0\n# 4    Ellen   NaN    CA   88.0\n# 5    Frank  30.0   NaN    NaN\nBy setting\nhow='any'\n, rows that contain at least one\nNaN\nare removed. Since the default value of\nhow\nis\n'any'\n, the result is the same even if omitted.\nprint\n(\ndf2\n.\ndropna\n(\nhow\n=\n'any'\n))\n#    name   age state  point\n# 3  Dave  68.0    TX   70.0\nprint\n(\ndf2\n.\ndropna\n())\n#    name   age state  point\n# 3  Dave  68.0    TX   70.0\nIf\naxis\nis set to\n1\nor\n'columns'\n, columns that contain at least one\nNaN\nare removed.\nprint\n(\ndf2\n.\ndropna\n(\naxis\n=\n1\n))\n#       name\n# 0    Alice\n# 2  Charlie\n# 3     Dave\n# 4    Ellen\n# 5    Frank\nRemove rows\/columns according to the number of non-missing values:\nthresh\nWith the\nthresh\nargument, you can remove rows and columns according to the number of non-missing values.\nFor example, if\nthresh=3\n, the rows that contain more than three non-missing values remain, and the other rows are removed.\nprint\n(\ndf\n.\ndropna\n(\nthresh\n=\n3\n))\n#     name   age state  point  other\n# 0  Alice  24.0    NY    NaN    NaN\n# 3   Dave  68.0    TX   70.0    NaN\n# 4  Ellen   NaN    CA   88.0    NaN\nIf\naxis\nis set to\n1\nor\n'columns'\n, columns are removed.\nprint\n(\ndf\n.\ndropna\n(\nthresh\n=\n3\n,\naxis\n=\n1\n))\n#       name   age state\n# 0    Alice  24.0    NY\n# 1      NaN   NaN   NaN\n# 2  Charlie   NaN    CA\n# 3     Dave  68.0    TX\n# 4    Ellen   NaN    CA\n# 5    Frank  30.0   NaN\nRemove based on specific rows\/columns:\nsubset\nIf you want to remove based on specific rows and columns, specify a list of rows\/columns labels (names) to the\nsubset\nargument of\ndropna()\n. Even if you want to set only one label, you need to specify it as a list, like\nsubset=['name']\n.\nSince the default is\nhow='any'\nand\naxis=0\n, rows with\nNaN\nin the columns specified by\nsubset\nare removed.\nprint\n(\ndf\n.\ndropna\n(\nsubset\n=\n[\n'age'\n]))\n#     name   age state  point  other\n# 0  Alice  24.0    NY    NaN    NaN\n# 3   Dave  68.0    TX   70.0    NaN\n# 5  Frank  30.0   NaN    NaN    NaN\nprint\n(\ndf\n.\ndropna\n(\nsubset\n=\n[\n'age'\n,\n'state'\n]))\n#     name   age state  point  other\n# 0  Alice  24.0    NY    NaN    NaN\n# 3   Dave  68.0    TX   70.0    NaN\nIf\nhow\nis set to\n'all'\n, rows with\nNaN\nin all specified columns are removed.\nprint\n(\ndf\n.\ndropna\n(\nsubset\n=\n[\n'age'\n,\n'state'\n],\nhow\n=\n'all'\n))\n#       name   age state  point  other\n# 0    Alice  24.0    NY    NaN    NaN\n# 2  Charlie   NaN    CA    NaN    NaN\n# 3     Dave  68.0    TX   70.0    NaN\n# 4    Ellen   NaN    CA   88.0    NaN\n# 5    Frank  30.0   NaN    NaN    NaN\nIf\naxis\nis set to\n1\nor\n'columns'\n, columns are removed.\nprint\n(\ndf\n.\ndropna\n(\nsubset\n=\n[\n0\n,\n4\n],\naxis\n=\n1\n))\n#       name state\n# 0    Alice    NY\n# 1      NaN   NaN\n# 2  Charlie    CA\n# 3     Dave    TX\n# 4    Ellen    CA\n# 5    Frank   NaN\nprint\n(\ndf\n.\ndropna\n(\nsubset\n=\n[\n0\n,\n4\n],\naxis\n=\n1\n,\nhow\n=\n'all'\n))\n#       name   age state  point\n# 0    Alice  24.0    NY    NaN\n# 1      NaN   NaN   NaN    NaN\n# 2  Charlie   NaN    CA    NaN\n# 3     Dave  68.0    TX   70.0\n# 4    Ellen   NaN    CA   88.0\n# 5    Frank  30.0   NaN    NaN\nAn error is raised if a non-existent row or column name is specified. An error is also raised if you set\naxis=1\nbut specify column names or set\naxis=0\n(default) but specify row names.\n# print(df.dropna(subset=['age', 'state', 'xxx']))\n# KeyError: ['xxx']\n# print(df.dropna(subset=['age', 'state'], axis=1))\n# KeyError: ['age', 'state']\nUpdate the original object:\ninplace\nAs shown in the examples above, by default, a new object is returned, and the original object is not changed, but if\ninplace=True\n, the original object itself is updated.\ndf\n.\ndropna\n(\nsubset\n=\n[\n'age'\n],\ninplace\n=\nTrue\n)\nprint\n(\ndf\n)\n#     name   age state  point  other\n# 0  Alice  24.0    NY    NaN    NaN\n# 3   Dave  68.0    TX   70.0    NaN\n# 5  Frank  30.0   NaN    NaN    NaN\nFor\npandas.Series\nThe only valid argument for\ndropna()\nof\npandas.Series\nis\ninplace\n. Since it is one-dimensional data, the elements with\nNaN\nare simply removed.\ns\n=\npd\n.\nread_csv\n(\n'data\/src\/sample_pandas_normal_nan.csv'\n)[\n'age'\n]\nprint\n(\ns\n)\n# 0    24.0\n# 1     NaN\n# 2     NaN\n# 3    68.0\n# 4     NaN\n# 5    30.0\n# Name: age, dtype: float64\nprint\n(\ns\n.\ndropna\n())\n# 0    24.0\n# 3    68.0\n# 5    30.0\n# Name: age, dtype: float64\ns\n.\ndropna\n(\ninplace\n=\nTrue\n)\nprint\n(\ns\n)\n# 0    24.0\n# 3    68.0\n# 5    30.0\n# Name: age, dtype: float64","attrs_markdown":"[note.nkmk.me](https:\/\/note.nkmk.me\/en\/)\n\n1. [Home](https:\/\/note.nkmk.me\/en\/)\n2. [Python](https:\/\/note.nkmk.me\/en\/python\/)\n3. [pandas](https:\/\/note.nkmk.me\/en\/pandas\/)\n\n# pandas: Remove NaN (missing values) with dropna()\nModified:\n\n2023-08-02\n\n\\| Tags: [Python](https:\/\/note.nkmk.me\/en\/python\/), [pandas](https:\/\/note.nkmk.me\/en\/pandas\/)\n\nYou can remove `NaN` from `pandas.DataFrame` and `pandas.Series` with the `dropna()` method.\n\n- [pandas.DataFrame.dropna — pandas 2.0.3 documentation](https:\/\/pandas.pydata.org\/pandas-docs\/stable\/reference\/api\/pandas.DataFrame.dropna.html)\n- [pandas.Series.dropna — pandas 2.0.3 documentation](https:\/\/pandas.pydata.org\/pandas-docs\/stable\/reference\/api\/pandas.Series.dropna.html)\n\nContents\n\n- [Remove rows\/columns where all elements are `NaN`: `how='all'`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#remove-rowscolumns-where-all-elements-are-nan-howall)\n- [Remove rows\/columns that contain at least one `NaN`: `how='any'` (default)](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#remove-rowscolumns-that-contain-at-least-one-nan-howany-default)\n- [Remove rows\/columns according to the number of non-missing values: `thresh`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#remove-rowscolumns-according-to-the-number-of-non-missing-values-thresh)\n- [Remove based on specific rows\/columns: `subset`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#remove-based-on-specific-rowscolumns-subset)\n- [Update the original object: `inplace`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#update-the-original-object-inplace)\n- [For `pandas.Series`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#for-pandasseries)\n\nWhile this article primarily deals with `NaN` (Not a Number), it's important to note that in pandas, `None` is also treated as a missing value.\n\n- [Missing values in pandas (nan, None, pd.NA)](https:\/\/note.nkmk.me\/en\/python-pandas-nan-none-na\/)\n\nSee the following article on extracting, replacing, and counting missing values.\n\n- [pandas: Find rows\/columns with NaN (missing values)](https:\/\/note.nkmk.me\/en\/python-pandas-nan-extract\/)\n- [pandas: Replace NaN (missing values) with fillna()](https:\/\/note.nkmk.me\/en\/python-pandas-nan-fillna\/)\n- [pandas: Detect and count NaN (missing values) with isnull(), isna()](https:\/\/note.nkmk.me\/en\/python-pandas-nan-judge-count\/)\n\nThe sample code in this article uses pandas version `2.0.3`. As an example, read a CSV file with missing values.\n\n- [sample\\_pandas\\_normal\\_nan.csv](https:\/\/raw.githubusercontent.com\/nkmk\/python-snippets\/217ee1c1fde5d816726f083185036af252a39647\/notebook\/data\/src\/sample_pandas_normal_nan.csv)\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L1-L14)\n## Remove rows\/columns where all elements are `NaN`: `how='all'`\nBy setting `how='all'`, rows where all elements are `NaN` are removed.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L16-L22)\n\nIf `axis` is set to `1` or `'columns'`, columns where all elements are `NaN` are removed.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L24-L31)\n\nNote that if `axis` is set to `0` or `'index'`, rows are removed. Since the default value of `axis` is `0`, rows are removed if omitted, as shown in the first example.\n\nIn former versions, both rows and columns were removed with `axis=[0, 1]`, but since version `1.0.0`, `axis` can no longer be specified with a list or tuple. If you want to remove both rows and columns, you can repeatedly apply `dropna()`.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L33-L42)\n## Remove rows\/columns that contain at least one `NaN`: `how='any'` (default)\nTo use as an example, remove rows and columns where all values are `NaN`.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L44-L51)\n\nBy setting `how='any'`, rows that contain at least one `NaN` are removed. Since the default value of `how` is `'any'`, the result is the same even if omitted.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L53-L59)\n\nIf `axis` is set to `1` or `'columns'`, columns that contain at least one `NaN` are removed.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L61-L67)\n## Remove rows\/columns according to the number of non-missing values: `thresh`\nWith the `thresh` argument, you can remove rows and columns according to the number of non-missing values.\n\nFor example, if `thresh=3`, the rows that contain more than three non-missing values remain, and the other rows are removed.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L69-L73)\n\nIf `axis` is set to `1` or `'columns'`, columns are removed.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L75-L82)\n## Remove based on specific rows\/columns: `subset`\nIf you want to remove based on specific rows and columns, specify a list of rows\/columns labels (names) to the `subset` argument of `dropna()`. Even if you want to set only one label, you need to specify it as a list, like `subset=['name']`.\n\nSince the default is `how='any'` and `axis=0`, rows with `NaN` in the columns specified by `subset` are removed.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L84-L93)\n\nIf `how` is set to `'all'`, rows with `NaN` in all specified columns are removed.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L95-L101)\n\nIf `axis` is set to `1` or `'columns'`, columns are removed.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L103-L119)\n\nAn error is raised if a non-existent row or column name is specified. An error is also raised if you set `axis=1` but specify column names or set `axis=0` (default) but specify row names.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L121-L125)\n## Update the original object: `inplace`\nAs shown in the examples above, by default, a new object is returned, and the original object is not changed, but if `inplace=True`, the original object itself is updated.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L127-L132)\n## For `pandas.Series`\nThe only valid argument for `dropna()` of `pandas.Series` is `inplace`. Since it is one-dimensional data, the elements with `NaN` are simply removed.\n\n```\n\n```\nsource: [pandas\\_nan\\_dropna.py](https:\/\/github.com\/nkmk\/python-snippets\/blob\/20611e7fec7b73380534460784071f486f57e1bd\/notebook\/pandas_nan_dropna.py#L134-L155)\n## Related Categories\n- [Python](https:\/\/note.nkmk.me\/en\/python\/)\n- [pandas](https:\/\/note.nkmk.me\/en\/pandas\/)\n## Related Articles\n- [pandas: Shuffle rows\/elements of DataFrame\/Series](https:\/\/note.nkmk.me\/en\/python-pandas-random-sort-shuffle\/)\n- [pandas: Replace Series values with map()](https:\/\/note.nkmk.me\/en\/python-pandas-map-replace\/)\n- [List of pandas articles](https:\/\/note.nkmk.me\/en\/python-pandas-post-summary\/)\n- [pandas: Iterate DataFrame with for loop (iterrows, itertuples, items)](https:\/\/note.nkmk.me\/en\/python-pandas-dataframe-for-iteration\/)\n- [pandas: Convert a list of dictionaries to DataFrame with json\\_normalize](https:\/\/note.nkmk.me\/en\/python-pandas-json-normalize\/)\n- [pandas: Detect and count NaN (missing values) with isnull(), isna()](https:\/\/note.nkmk.me\/en\/python-pandas-nan-judge-count\/)\n- [pandas: Get first\/last n rows of DataFrame with head() and tail()](https:\/\/note.nkmk.me\/en\/python-pandas-head-tail\/)\n- [pandas: Get and set options for display, data behavior, etc.](https:\/\/note.nkmk.me\/en\/python-pandas-option-setting\/)\n- [pandas: Merge DataFrame with merge(), join() (INNER, OUTER JOIN)](https:\/\/note.nkmk.me\/en\/python-pandas-merge-join\/)\n- [pandas: Sort DataFrame\/Series with sort\\_values(), sort\\_index()](https:\/\/note.nkmk.me\/en\/python-pandas-sort-values-sort-index\/)\n- [pandas: Get unique values and their counts in a column](https:\/\/note.nkmk.me\/en\/python-pandas-value-counts\/)\n- [pandas: Views and copies in DataFrame](https:\/\/note.nkmk.me\/en\/python-pandas-view-copy\/)\n- [pandas: Read CSV into DataFrame with read\\_csv()](https:\/\/note.nkmk.me\/en\/python-pandas-read-csv-tsv\/)\n- [pandas: Select rows\/columns by index (numbers and names)](https:\/\/note.nkmk.me\/en\/python-pandas-index-row-column\/)\n- [pandas: Slice substrings from each element in columns](https:\/\/note.nkmk.me\/en\/python-pandas-str-slice\/)\n\nSearch\n\nCategories\n\n- [Python](https:\/\/note.nkmk.me\/en\/python\/)\n  - [NumPy](https:\/\/note.nkmk.me\/en\/numpy\/)\n  - [OpenCV](https:\/\/note.nkmk.me\/en\/opencv\/)\n  - [pandas](https:\/\/note.nkmk.me\/en\/pandas\/)\n  - [Pillow](https:\/\/note.nkmk.me\/en\/pillow\/)\n  - [pip](https:\/\/note.nkmk.me\/en\/pip\/)\n  - [scikit-image](https:\/\/note.nkmk.me\/en\/scikit-image\/)\n  - [uv](https:\/\/note.nkmk.me\/en\/uv\/)\n\n- [Git](https:\/\/note.nkmk.me\/en\/git\/)\n- [Jupyter Notebook](https:\/\/note.nkmk.me\/en\/jupyter-notebook\/)\n- [Mac](https:\/\/note.nkmk.me\/en\/mac\/)\n- [Windows](https:\/\/note.nkmk.me\/en\/windows\/)\n- [Image Processing](https:\/\/note.nkmk.me\/en\/image-processing\/)\n- [File](https:\/\/note.nkmk.me\/en\/file\/)\n- [CSV](https:\/\/note.nkmk.me\/en\/csv\/)\n- [JSON](https:\/\/note.nkmk.me\/en\/json\/)\n- [PDF](https:\/\/note.nkmk.me\/en\/pdf\/)\n- [Date and time](https:\/\/note.nkmk.me\/en\/date-and-time\/)\n- [String](https:\/\/note.nkmk.me\/en\/string\/)\n- [Regex](https:\/\/note.nkmk.me\/en\/regex\/)\n- [Numeric](https:\/\/note.nkmk.me\/en\/numeric\/)\n- [Dictionary](https:\/\/note.nkmk.me\/en\/dictionary\/)\n- [List](https:\/\/note.nkmk.me\/en\/list\/)\n- [Error handling](https:\/\/note.nkmk.me\/en\/error-handling\/)\n- [Mathematics](https:\/\/note.nkmk.me\/en\/mathematics\/)\n- [Summary](https:\/\/note.nkmk.me\/en\/summary\/)\n\nAbout\n\n- GitHub: [nkmk](https:\/\/github.com\/nkmk)\n\nRelated Articles\n\n- [pandas: Shuffle rows\/elements of DataFrame\/Series](https:\/\/note.nkmk.me\/en\/python-pandas-random-sort-shuffle\/)\n- [pandas: Replace Series values with map()](https:\/\/note.nkmk.me\/en\/python-pandas-map-replace\/)\n- [List of pandas articles](https:\/\/note.nkmk.me\/en\/python-pandas-post-summary\/)\n- [pandas: Iterate DataFrame with for loop (iterrows, itertuples, items)](https:\/\/note.nkmk.me\/en\/python-pandas-dataframe-for-iteration\/)\n- [pandas: Convert a list of dictionaries to DataFrame with json\\_normalize](https:\/\/note.nkmk.me\/en\/python-pandas-json-normalize\/)\n\n- English \/ [Japanese](https:\/\/note.nkmk.me\/)\n- \\|\n- [Disclaimer](https:\/\/note.nkmk.me\/en\/disclaimer\/)\n- [Privacy policy](https:\/\/note.nkmk.me\/en\/privacy-policy\/)\n- [GitHub](https:\/\/github.com\/nkmk)\n- ©[nkmk.me](https:\/\/nkmk.me\/)","attrs_readable_markdown":"You can remove `NaN` from `pandas.DataFrame` and `pandas.Series` with the `dropna()` method.\n\n- [pandas.DataFrame.dropna — pandas 2.0.3 documentation](https:\/\/pandas.pydata.org\/pandas-docs\/stable\/reference\/api\/pandas.DataFrame.dropna.html)\n- [pandas.Series.dropna — pandas 2.0.3 documentation](https:\/\/pandas.pydata.org\/pandas-docs\/stable\/reference\/api\/pandas.Series.dropna.html)\n\nContents\n\n- [Remove rows\/columns where all elements are `NaN`: `how='all'`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#remove-rowscolumns-where-all-elements-are-nan-howall)\n- [Remove rows\/columns that contain at least one `NaN`: `how='any'` (default)](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#remove-rowscolumns-that-contain-at-least-one-nan-howany-default)\n- [Remove rows\/columns according to the number of non-missing values: `thresh`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#remove-rowscolumns-according-to-the-number-of-non-missing-values-thresh)\n- [Remove based on specific rows\/columns: `subset`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#remove-based-on-specific-rowscolumns-subset)\n- [Update the original object: `inplace`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#update-the-original-object-inplace)\n- [For `pandas.Series`](https:\/\/note.nkmk.me\/en\/python-pandas-nan-dropna\/#for-pandasseries)\n\nWhile this article primarily deals with `NaN` (Not a Number), it's important to note that in pandas, `None` is also treated as a missing value.\n\n- [Missing values in pandas (nan, None, pd.NA)](https:\/\/note.nkmk.me\/en\/python-pandas-nan-none-na\/)\n\nSee the following article on extracting, replacing, and counting missing values.\n\n- [pandas: Find rows\/columns with NaN (missing values)](https:\/\/note.nkmk.me\/en\/python-pandas-nan-extract\/)\n- [pandas: Replace NaN (missing values) with fillna()](https:\/\/note.nkmk.me\/en\/python-pandas-nan-fillna\/)\n- [pandas: Detect and count NaN (missing values) with isnull(), isna()](https:\/\/note.nkmk.me\/en\/python-pandas-nan-judge-count\/)\n\nThe sample code in this article uses pandas version `2.0.3`. As an example, read a CSV file with missing values.\n\n- [sample\\_pandas\\_normal\\_nan.csv](https:\/\/raw.githubusercontent.com\/nkmk\/python-snippets\/217ee1c1fde5d816726f083185036af252a39647\/notebook\/data\/src\/sample_pandas_normal_nan.csv)\n```\n\n```\n## Remove rows\/columns where all elements are `NaN`: `how='all'`\nBy setting `how='all'`, rows where all elements are `NaN` are removed.\n```\n\n```\nIf `axis` is set to `1` or `'columns'`, columns where all elements are `NaN` are removed.\n```\n\n```\nNote that if `axis` is set to `0` or `'index'`, rows are removed. Since the default value of `axis` is `0`, rows are removed if omitted, as shown in the first example.\n\nIn former versions, both rows and columns were removed with `axis=[0, 1]`, but since version `1.0.0`, `axis` can no longer be specified with a list or tuple. If you want to remove both rows and columns, you can repeatedly apply `dropna()`.\n```\n\n```\n## Remove rows\/columns that contain at least one `NaN`: `how='any'` (default)\nTo use as an example, remove rows and columns where all values are `NaN`.\n```\n\n```\nBy setting `how='any'`, rows that contain at least one `NaN` are removed. Since the default value of `how` is `'any'`, the result is the same even if omitted.\n```\n\n```\nIf `axis` is set to `1` or `'columns'`, columns that contain at least one `NaN` are removed.\n```\n\n```\n## Remove rows\/columns according to the number of non-missing values: `thresh`\nWith the `thresh` argument, you can remove rows and columns according to the number of non-missing values.\n\nFor example, if `thresh=3`, the rows that contain more than three non-missing values remain, and the other rows are removed.\n```\n\n```\nIf `axis` is set to `1` or `'columns'`, columns are removed.\n```\n\n```\n## Remove based on specific rows\/columns: `subset`\nIf you want to remove based on specific rows and columns, specify a list of rows\/columns labels (names) to the `subset` argument of `dropna()`. Even if you want to set only one label, you need to specify it as a list, like `subset=['name']`.\n\nSince the default is `how='any'` and `axis=0`, rows with `NaN` in the columns specified by `subset` are removed.\n```\n\n```\nIf `how` is set to `'all'`, rows with `NaN` in all specified columns are removed.\n```\n\n```\nIf `axis` is set to `1` or `'columns'`, columns are removed.\n```\n\n```\nAn error is raised if a non-existent row or column name is specified. An error is also raised if you set `axis=1` but specify column names or set `axis=0` (default) but specify row names.\n```\n\n```\n## Update the original object: `inplace`\nAs shown in the examples above, by default, a new object is returned, and the original object is not changed, but if `inplace=True`, the original object itself is updated.\n```\n\n```\n## For `pandas.Series`\nThe only valid argument for `dropna()` of `pandas.Series` is `inplace`. Since it is one-dimensional data, the elements with `NaN` are simply removed.\n```\n\n```","meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

16 days ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	0.5 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	https://note.nkmk.me/en/python-pandas-nan-dropna/
Last Crawled	2026-03-27 12:27:53 (16 days ago)
First Indexed	2022-02-13 07:19:01 (4 years ago)
HTTP Status Code	200
Meta Title	pandas: Remove NaN (missing values) with dropna() \| note.nkmk.me
Meta Description	You can remove NaN from pandas.DataFrame and pandas.Series with the dropna() method. pandas.DataFrame.dropna — pandas 2.0.3 documentation pandas.Series.dropna — pandas 2.0.3 documentation Remove row ...
Meta Canonical	null
Boilerpipe Text	You can remove NaN from pandas.DataFrame and pandas.Series with the dropna() method. pandas.DataFrame.dropna — pandas 2.0.3 documentation pandas.Series.dropna — pandas 2.0.3 documentation Contents Remove rows/columns where all elements are NaN : how='all' Remove rows/columns that contain at least one NaN : how='any' (default) Remove rows/columns according to the number of non-missing values: thresh Remove based on specific rows/columns: subset Update the original object: inplace For pandas.Series While this article primarily deals with NaN (Not a Number), it's important to note that in pandas, None is also treated as a missing value. Missing values in pandas (nan, None, pd.NA) See the following article on extracting, replacing, and counting missing values. pandas: Find rows/columns with NaN (missing values) pandas: Replace NaN (missing values) with fillna() pandas: Detect and count NaN (missing values) with isnull(), isna() The sample code in this article uses pandas version 2.0.3 . As an example, read a CSV file with missing values. sample_pandas_normal_nan.csv import pandas as pd print ( pd . __version__ ) # 2.0.3 df = pd . read_csv ( 'data/src/sample_pandas_normal_nan.csv' ) print ( df ) # name age state point other # 0 Alice 24.0 NY NaN NaN # 1 NaN NaN NaN NaN NaN # 2 Charlie NaN CA NaN NaN # 3 Dave 68.0 TX 70.0 NaN # 4 Ellen NaN CA 88.0 NaN # 5 Frank 30.0 NaN NaN NaN Remove rows/columns where all elements are NaN : how='all' By setting how='all' , rows where all elements are NaN are removed. print ( df . dropna ( how = 'all' )) # name age state point other # 0 Alice 24.0 NY NaN NaN # 2 Charlie NaN CA NaN NaN # 3 Dave 68.0 TX 70.0 NaN # 4 Ellen NaN CA 88.0 NaN # 5 Frank 30.0 NaN NaN NaN If axis is set to 1 or 'columns' , columns where all elements are NaN are removed. print ( df . dropna ( how = 'all' , axis = 1 )) # name age state point # 0 Alice 24.0 NY NaN # 1 NaN NaN NaN NaN # 2 Charlie NaN CA NaN # 3 Dave 68.0 TX 70.0 # 4 Ellen NaN CA 88.0 # 5 Frank 30.0 NaN NaN Note that if axis is set to 0 or 'index' , rows are removed. Since the default value of axis is 0 , rows are removed if omitted, as shown in the first example. In former versions, both rows and columns were removed with axis=[0, 1] , but since version 1.0.0 , axis can no longer be specified with a list or tuple. If you want to remove both rows and columns, you can repeatedly apply dropna() . # print(df.dropna(how='all', axis=[0, 1])) # TypeError: supplying multiple axes to axis is no longer supported. print ( df . dropna ( how = 'all' ) . dropna ( how = 'all' , axis = 1 )) # name age state point # 0 Alice 24.0 NY NaN # 2 Charlie NaN CA NaN # 3 Dave 68.0 TX 70.0 # 4 Ellen NaN CA 88.0 # 5 Frank 30.0 NaN NaN Remove rows/columns that contain at least one NaN : how='any' (default) To use as an example, remove rows and columns where all values are NaN . df2 = df . dropna ( how = 'all' ) . dropna ( how = 'all' , axis = 1 ) print ( df2 ) # name age state point # 0 Alice 24.0 NY NaN # 2 Charlie NaN CA NaN # 3 Dave 68.0 TX 70.0 # 4 Ellen NaN CA 88.0 # 5 Frank 30.0 NaN NaN By setting how='any' , rows that contain at least one NaN are removed. Since the default value of how is 'any' , the result is the same even if omitted. print ( df2 . dropna ( how = 'any' )) # name age state point # 3 Dave 68.0 TX 70.0 print ( df2 . dropna ()) # name age state point # 3 Dave 68.0 TX 70.0 If axis is set to 1 or 'columns' , columns that contain at least one NaN are removed. print ( df2 . dropna ( axis = 1 )) # name # 0 Alice # 2 Charlie # 3 Dave # 4 Ellen # 5 Frank Remove rows/columns according to the number of non-missing values: thresh With the thresh argument, you can remove rows and columns according to the number of non-missing values. For example, if thresh=3 , the rows that contain more than three non-missing values remain, and the other rows are removed. print ( df . dropna ( thresh = 3 )) # name age state point other # 0 Alice 24.0 NY NaN NaN # 3 Dave 68.0 TX 70.0 NaN # 4 Ellen NaN CA 88.0 NaN If axis is set to 1 or 'columns' , columns are removed. print ( df . dropna ( thresh = 3 , axis = 1 )) # name age state # 0 Alice 24.0 NY # 1 NaN NaN NaN # 2 Charlie NaN CA # 3 Dave 68.0 TX # 4 Ellen NaN CA # 5 Frank 30.0 NaN Remove based on specific rows/columns: subset If you want to remove based on specific rows and columns, specify a list of rows/columns labels (names) to the subset argument of dropna() . Even if you want to set only one label, you need to specify it as a list, like subset=['name'] . Since the default is how='any' and axis=0 , rows with NaN in the columns specified by subset are removed. print ( df . dropna ( subset = [ 'age' ])) # name age state point other # 0 Alice 24.0 NY NaN NaN # 3 Dave 68.0 TX 70.0 NaN # 5 Frank 30.0 NaN NaN NaN print ( df . dropna ( subset = [ 'age' , 'state' ])) # name age state point other # 0 Alice 24.0 NY NaN NaN # 3 Dave 68.0 TX 70.0 NaN If how is set to 'all' , rows with NaN in all specified columns are removed. print ( df . dropna ( subset = [ 'age' , 'state' ], how = 'all' )) # name age state point other # 0 Alice 24.0 NY NaN NaN # 2 Charlie NaN CA NaN NaN # 3 Dave 68.0 TX 70.0 NaN # 4 Ellen NaN CA 88.0 NaN # 5 Frank 30.0 NaN NaN NaN If axis is set to 1 or 'columns' , columns are removed. print ( df . dropna ( subset = [ 0 , 4 ], axis = 1 )) # name state # 0 Alice NY # 1 NaN NaN # 2 Charlie CA # 3 Dave TX # 4 Ellen CA # 5 Frank NaN print ( df . dropna ( subset = [ 0 , 4 ], axis = 1 , how = 'all' )) # name age state point # 0 Alice 24.0 NY NaN # 1 NaN NaN NaN NaN # 2 Charlie NaN CA NaN # 3 Dave 68.0 TX 70.0 # 4 Ellen NaN CA 88.0 # 5 Frank 30.0 NaN NaN An error is raised if a non-existent row or column name is specified. An error is also raised if you set axis=1 but specify column names or set axis=0 (default) but specify row names. # print(df.dropna(subset=['age', 'state', 'xxx'])) # KeyError: ['xxx'] # print(df.dropna(subset=['age', 'state'], axis=1)) # KeyError: ['age', 'state'] Update the original object: inplace As shown in the examples above, by default, a new object is returned, and the original object is not changed, but if inplace=True , the original object itself is updated. df . dropna ( subset = [ 'age' ], inplace = True ) print ( df ) # name age state point other # 0 Alice 24.0 NY NaN NaN # 3 Dave 68.0 TX 70.0 NaN # 5 Frank 30.0 NaN NaN NaN For pandas.Series The only valid argument for dropna() of pandas.Series is inplace . Since it is one-dimensional data, the elements with NaN are simply removed. s = pd . read_csv ( 'data/src/sample_pandas_normal_nan.csv' )[ 'age' ] print ( s ) # 0 24.0 # 1 NaN # 2 NaN # 3 68.0 # 4 NaN # 5 30.0 # Name: age, dtype: float64 print ( s . dropna ()) # 0 24.0 # 3 68.0 # 5 30.0 # Name: age, dtype: float64 s . dropna ( inplace = True ) print ( s ) # 0 24.0 # 3 68.0 # 5 30.0 # Name: age, dtype: float64
Markdown	[note.nkmk.me](https://note.nkmk.me/en/) 1. [Home](https://note.nkmk.me/en/) 2. [Python](https://note.nkmk.me/en/python/) 3. [pandas](https://note.nkmk.me/en/pandas/) # pandas: Remove NaN (missing values) with dropna() Modified: 2023-08-02 \\| Tags: [Python](https://note.nkmk.me/en/python/), [pandas](https://note.nkmk.me/en/pandas/) You can remove `NaN` from `pandas.DataFrame` and `pandas.Series` with the `dropna()` method. - [pandas.DataFrame.dropna — pandas 2.0.3 documentation](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.dropna.html) - [pandas.Series.dropna — pandas 2.0.3 documentation](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.dropna.html) Contents - [Remove rows/columns where all elements are `NaN`: `how='all'`](https://note.nkmk.me/en/python-pandas-nan-dropna/#remove-rowscolumns-where-all-elements-are-nan-howall) - [Remove rows/columns that contain at least one `NaN`: `how='any'` (default)](https://note.nkmk.me/en/python-pandas-nan-dropna/#remove-rowscolumns-that-contain-at-least-one-nan-howany-default) - [Remove rows/columns according to the number of non-missing values: `thresh`](https://note.nkmk.me/en/python-pandas-nan-dropna/#remove-rowscolumns-according-to-the-number-of-non-missing-values-thresh) - [Remove based on specific rows/columns: `subset`](https://note.nkmk.me/en/python-pandas-nan-dropna/#remove-based-on-specific-rowscolumns-subset) - [Update the original object: `inplace`](https://note.nkmk.me/en/python-pandas-nan-dropna/#update-the-original-object-inplace) - [For `pandas.Series`](https://note.nkmk.me/en/python-pandas-nan-dropna/#for-pandasseries) While this article primarily deals with `NaN` (Not a Number), it's important to note that in pandas, `None` is also treated as a missing value. - [Missing values in pandas (nan, None, pd.NA)](https://note.nkmk.me/en/python-pandas-nan-none-na/) See the following article on extracting, replacing, and counting missing values. - [pandas: Find rows/columns with NaN (missing values)](https://note.nkmk.me/en/python-pandas-nan-extract/) - [pandas: Replace NaN (missing values) with fillna()](https://note.nkmk.me/en/python-pandas-nan-fillna/) - [pandas: Detect and count NaN (missing values) with isnull(), isna()](https://note.nkmk.me/en/python-pandas-nan-judge-count/) The sample code in this article uses pandas version `2.0.3`. As an example, read a CSV file with missing values. - [sample\_pandas\_normal\_nan.csv](https://raw.githubusercontent.com/nkmk/python-snippets/217ee1c1fde5d816726f083185036af252a39647/notebook/data/src/sample_pandas_normal_nan.csv) ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L1-L14) ## Remove rows/columns where all elements are `NaN`: `how='all'` By setting `how='all'`, rows where all elements are `NaN` are removed. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L16-L22) If `axis` is set to `1` or `'columns'`, columns where all elements are `NaN` are removed. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L24-L31) Note that if `axis` is set to `0` or `'index'`, rows are removed. Since the default value of `axis` is `0`, rows are removed if omitted, as shown in the first example. In former versions, both rows and columns were removed with `axis=[0, 1]`, but since version `1.0.0`, `axis` can no longer be specified with a list or tuple. If you want to remove both rows and columns, you can repeatedly apply `dropna()`. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L33-L42) ## Remove rows/columns that contain at least one `NaN`: `how='any'` (default) To use as an example, remove rows and columns where all values are `NaN`. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L44-L51) By setting `how='any'`, rows that contain at least one `NaN` are removed. Since the default value of `how` is `'any'`, the result is the same even if omitted. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L53-L59) If `axis` is set to `1` or `'columns'`, columns that contain at least one `NaN` are removed. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L61-L67) ## Remove rows/columns according to the number of non-missing values: `thresh` With the `thresh` argument, you can remove rows and columns according to the number of non-missing values. For example, if `thresh=3`, the rows that contain more than three non-missing values remain, and the other rows are removed. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L69-L73) If `axis` is set to `1` or `'columns'`, columns are removed. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L75-L82) ## Remove based on specific rows/columns: `subset` If you want to remove based on specific rows and columns, specify a list of rows/columns labels (names) to the `subset` argument of `dropna()`. Even if you want to set only one label, you need to specify it as a list, like `subset=['name']`. Since the default is `how='any'` and `axis=0`, rows with `NaN` in the columns specified by `subset` are removed. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L84-L93) If `how` is set to `'all'`, rows with `NaN` in all specified columns are removed. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L95-L101) If `axis` is set to `1` or `'columns'`, columns are removed. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L103-L119) An error is raised if a non-existent row or column name is specified. An error is also raised if you set `axis=1` but specify column names or set `axis=0` (default) but specify row names. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L121-L125) ## Update the original object: `inplace` As shown in the examples above, by default, a new object is returned, and the original object is not changed, but if `inplace=True`, the original object itself is updated. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L127-L132) ## For `pandas.Series` The only valid argument for `dropna()` of `pandas.Series` is `inplace`. Since it is one-dimensional data, the elements with `NaN` are simply removed. ``` ``` source: [pandas\_nan\_dropna.py](https://github.com/nkmk/python-snippets/blob/20611e7fec7b73380534460784071f486f57e1bd/notebook/pandas_nan_dropna.py#L134-L155) ## Related Categories - [Python](https://note.nkmk.me/en/python/) - [pandas](https://note.nkmk.me/en/pandas/) ## Related Articles - [pandas: Shuffle rows/elements of DataFrame/Series](https://note.nkmk.me/en/python-pandas-random-sort-shuffle/) - [pandas: Replace Series values with map()](https://note.nkmk.me/en/python-pandas-map-replace/) - [List of pandas articles](https://note.nkmk.me/en/python-pandas-post-summary/) - [pandas: Iterate DataFrame with for loop (iterrows, itertuples, items)](https://note.nkmk.me/en/python-pandas-dataframe-for-iteration/) - [pandas: Convert a list of dictionaries to DataFrame with json\_normalize](https://note.nkmk.me/en/python-pandas-json-normalize/) - [pandas: Detect and count NaN (missing values) with isnull(), isna()](https://note.nkmk.me/en/python-pandas-nan-judge-count/) - [pandas: Get first/last n rows of DataFrame with head() and tail()](https://note.nkmk.me/en/python-pandas-head-tail/) - [pandas: Get and set options for display, data behavior, etc.](https://note.nkmk.me/en/python-pandas-option-setting/) - [pandas: Merge DataFrame with merge(), join() (INNER, OUTER JOIN)](https://note.nkmk.me/en/python-pandas-merge-join/) - [pandas: Sort DataFrame/Series with sort\_values(), sort\_index()](https://note.nkmk.me/en/python-pandas-sort-values-sort-index/) - [pandas: Get unique values and their counts in a column](https://note.nkmk.me/en/python-pandas-value-counts/) - [pandas: Views and copies in DataFrame](https://note.nkmk.me/en/python-pandas-view-copy/) - [pandas: Read CSV into DataFrame with read\_csv()](https://note.nkmk.me/en/python-pandas-read-csv-tsv/) - [pandas: Select rows/columns by index (numbers and names)](https://note.nkmk.me/en/python-pandas-index-row-column/) - [pandas: Slice substrings from each element in columns](https://note.nkmk.me/en/python-pandas-str-slice/) Search Categories - [Python](https://note.nkmk.me/en/python/) - [NumPy](https://note.nkmk.me/en/numpy/) - [OpenCV](https://note.nkmk.me/en/opencv/) - [pandas](https://note.nkmk.me/en/pandas/) - [Pillow](https://note.nkmk.me/en/pillow/) - [pip](https://note.nkmk.me/en/pip/) - [scikit-image](https://note.nkmk.me/en/scikit-image/) - [uv](https://note.nkmk.me/en/uv/) - [Git](https://note.nkmk.me/en/git/) - [Jupyter Notebook](https://note.nkmk.me/en/jupyter-notebook/) - [Mac](https://note.nkmk.me/en/mac/) - [Windows](https://note.nkmk.me/en/windows/) - [Image Processing](https://note.nkmk.me/en/image-processing/) - [File](https://note.nkmk.me/en/file/) - [CSV](https://note.nkmk.me/en/csv/) - [JSON](https://note.nkmk.me/en/json/) - [PDF](https://note.nkmk.me/en/pdf/) - [Date and time](https://note.nkmk.me/en/date-and-time/) - [String](https://note.nkmk.me/en/string/) - [Regex](https://note.nkmk.me/en/regex/) - [Numeric](https://note.nkmk.me/en/numeric/) - [Dictionary](https://note.nkmk.me/en/dictionary/) - [List](https://note.nkmk.me/en/list/) - [Error handling](https://note.nkmk.me/en/error-handling/) - [Mathematics](https://note.nkmk.me/en/mathematics/) - [Summary](https://note.nkmk.me/en/summary/) About - GitHub: [nkmk](https://github.com/nkmk) Related Articles - [pandas: Shuffle rows/elements of DataFrame/Series](https://note.nkmk.me/en/python-pandas-random-sort-shuffle/) - [pandas: Replace Series values with map()](https://note.nkmk.me/en/python-pandas-map-replace/) - [List of pandas articles](https://note.nkmk.me/en/python-pandas-post-summary/) - [pandas: Iterate DataFrame with for loop (iterrows, itertuples, items)](https://note.nkmk.me/en/python-pandas-dataframe-for-iteration/) - [pandas: Convert a list of dictionaries to DataFrame with json\_normalize](https://note.nkmk.me/en/python-pandas-json-normalize/) - English / [Japanese](https://note.nkmk.me/) - \\| - [Disclaimer](https://note.nkmk.me/en/disclaimer/) - [Privacy policy](https://note.nkmk.me/en/privacy-policy/) - [GitHub](https://github.com/nkmk) - ©[nkmk.me](https://nkmk.me/)
Readable Markdown	You can remove `NaN` from `pandas.DataFrame` and `pandas.Series` with the `dropna()` method. - [pandas.DataFrame.dropna — pandas 2.0.3 documentation](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.dropna.html) - [pandas.Series.dropna — pandas 2.0.3 documentation](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.dropna.html) Contents - [Remove rows/columns where all elements are `NaN`: `how='all'`](https://note.nkmk.me/en/python-pandas-nan-dropna/#remove-rowscolumns-where-all-elements-are-nan-howall) - [Remove rows/columns that contain at least one `NaN`: `how='any'` (default)](https://note.nkmk.me/en/python-pandas-nan-dropna/#remove-rowscolumns-that-contain-at-least-one-nan-howany-default) - [Remove rows/columns according to the number of non-missing values: `thresh`](https://note.nkmk.me/en/python-pandas-nan-dropna/#remove-rowscolumns-according-to-the-number-of-non-missing-values-thresh) - [Remove based on specific rows/columns: `subset`](https://note.nkmk.me/en/python-pandas-nan-dropna/#remove-based-on-specific-rowscolumns-subset) - [Update the original object: `inplace`](https://note.nkmk.me/en/python-pandas-nan-dropna/#update-the-original-object-inplace) - [For `pandas.Series`](https://note.nkmk.me/en/python-pandas-nan-dropna/#for-pandasseries) While this article primarily deals with `NaN` (Not a Number), it's important to note that in pandas, `None` is also treated as a missing value. - [Missing values in pandas (nan, None, pd.NA)](https://note.nkmk.me/en/python-pandas-nan-none-na/) See the following article on extracting, replacing, and counting missing values. - [pandas: Find rows/columns with NaN (missing values)](https://note.nkmk.me/en/python-pandas-nan-extract/) - [pandas: Replace NaN (missing values) with fillna()](https://note.nkmk.me/en/python-pandas-nan-fillna/) - [pandas: Detect and count NaN (missing values) with isnull(), isna()](https://note.nkmk.me/en/python-pandas-nan-judge-count/) The sample code in this article uses pandas version `2.0.3`. As an example, read a CSV file with missing values. - [sample\_pandas\_normal\_nan.csv](https://raw.githubusercontent.com/nkmk/python-snippets/217ee1c1fde5d816726f083185036af252a39647/notebook/data/src/sample_pandas_normal_nan.csv) ``` ``` ## Remove rows/columns where all elements are `NaN`: `how='all'` By setting `how='all'`, rows where all elements are `NaN` are removed. ``` ``` If `axis` is set to `1` or `'columns'`, columns where all elements are `NaN` are removed. ``` ``` Note that if `axis` is set to `0` or `'index'`, rows are removed. Since the default value of `axis` is `0`, rows are removed if omitted, as shown in the first example. In former versions, both rows and columns were removed with `axis=[0, 1]`, but since version `1.0.0`, `axis` can no longer be specified with a list or tuple. If you want to remove both rows and columns, you can repeatedly apply `dropna()`. ``` ``` ## Remove rows/columns that contain at least one `NaN`: `how='any'` (default) To use as an example, remove rows and columns where all values are `NaN`. ``` ``` By setting `how='any'`, rows that contain at least one `NaN` are removed. Since the default value of `how` is `'any'`, the result is the same even if omitted. ``` ``` If `axis` is set to `1` or `'columns'`, columns that contain at least one `NaN` are removed. ``` ``` ## Remove rows/columns according to the number of non-missing values: `thresh` With the `thresh` argument, you can remove rows and columns according to the number of non-missing values. For example, if `thresh=3`, the rows that contain more than three non-missing values remain, and the other rows are removed. ``` ``` If `axis` is set to `1` or `'columns'`, columns are removed. ``` ``` ## Remove based on specific rows/columns: `subset` If you want to remove based on specific rows and columns, specify a list of rows/columns labels (names) to the `subset` argument of `dropna()`. Even if you want to set only one label, you need to specify it as a list, like `subset=['name']`. Since the default is `how='any'` and `axis=0`, rows with `NaN` in the columns specified by `subset` are removed. ``` ``` If `how` is set to `'all'`, rows with `NaN` in all specified columns are removed. ``` ``` If `axis` is set to `1` or `'columns'`, columns are removed. ``` ``` An error is raised if a non-existent row or column name is specified. An error is also raised if you set `axis=1` but specify column names or set `axis=0` (default) but specify row names. ``` ``` ## Update the original object: `inplace` As shown in the examples above, by default, a new object is returned, and the original object is not changed, but if `inplace=True`, the original object itself is updated. ``` ``` ## For `pandas.Series` The only valid argument for `dropna()` of `pandas.Series` is `inplace`. Since it is one-dimensional data, the elements with `NaN` are simply removed. ``` ```
Shard	13 (laksa)
Root Hash	14415757146955323613
Unparsed URL	me,nkmk!note,/en/python-pandas-nan-dropna/ s443