{"id":3405,"date":"2025-10-31T10:59:22","date_gmt":"2025-10-31T09:59:22","guid":{"rendered":"https:\/\/neuraldesigner.com\/blog\/missing-values\/"},"modified":"2025-11-27T15:17:57","modified_gmt":"2025-11-27T14:17:57","slug":"missing-values","status":"publish","type":"blog","link":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/","title":{"rendered":"Handle missing values in machine learning"},"content":{"rendered":"

In this post, we explain the primary methods for handling missing values in machine learning and when to use each one.<\/p>\n

A common challenge when applying neural networks<\/a> to real-world problems is that datasets are often incomplete.<\/p>\n

To address this, you can either remove the samples with missing values or replace them with simple statistics such as the variable\u2019s mean.<\/p>\n

However, no single method is always the best choice, so you should study your dataset carefully to decide which approach gives the best results.<\/p>\n

\n

Contents<\/h1>\n
    \n
  1. \n
      \n
    1. Samples unusing<\/a><\/li>\n
    2. Data imputation<\/a><\/li>\n
    3. Time series data interpolation<\/a><\/li>\n
    4. Conclusions<\/a><\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

      <\/p>\n<\/section>\n

      \n

      Introduction<\/h2>\n

      Usually, missing values are denoted by a label in the data set.<\/p>\n

      Some standard labels used for representing missing values are NA<\/i> (not available), NaN<\/i> (not a number), Unknown<\/i>, or ?<\/i>.<\/p>\n

      Using numeric values, such as -999<\/b>, is not recommended as they could be mistaken for real values.<\/p>\n

      Throughout this work, we will choose the label NA <\/i>to denote a missing value.<\/p>\n

      2. Samples unusing<\/h2>\n

      This method sets samples as unused if they have missing values, thereby excluding them from the analysis in the machine learning model.<\/p>\n

      ui<\/sub> = unused, if ui<\/sub> contains NA, i=1,…,p<\/em>.<\/p>\n

      We can use it when the number of samples in the data set is large, and the number of missing values is negligible compared to the number of samples.<\/p>\n

      Example: Samples unusing<\/h3>\n

      A hospital has conducted a study to predict which treatment produces the best survival in patients with colon cancer after surgery.<\/p>\n

      How well a patient does after surgery depends on how much residual cancer remains.<\/p>\n

      This example examines data from a randomized controlled trial (RCT) measuring the effect of a particular drug combination on colon cancer.<\/p>\n

      The study data set has samples from 607 patients and 12 variables.<\/p>\n\n\n\n\n\n\n\n
      ID<\/th>\nSex<\/th>\nAge<\/th>\nObstruction<\/th>\nPerforation<\/th>\n…<\/th>\nOutcome<\/th>\n<\/tr>\n
      1<\/td>\nmale<\/td>\n43<\/td>\nno<\/td>\nno<\/td>\n…<\/td>\ndie<\/td>\n<\/tr>\n
      2<\/td>\nmale<\/td>\n63<\/td>\nno<\/td>\nno<\/td>\n…<\/td>\nsurvive<\/td>\n<\/tr>\n
      …<\/td>\n…<\/td>\n…<\/td>\n…<\/td>\n…<\/td>\n…<\/td>\n…<\/td>\n<\/tr>\n
      928<\/td>\nfemale<\/td>\n48<\/td>\nyes<\/td>\nno<\/td>\n…<\/td>\nsurvive<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

      The total number of missing values is 17.<\/p>\n\n\n\n\n
      Missing values<\/th>\nMissing samples<\/th>\n<\/tr>\n
      17 (0%)<\/td>\n17 (2%)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

      Since the number of missing values is negligible in the number of samples we have, we can eliminate the samples containing missing values from the analysis.<\/p>\n

      After removing the samples containing missing values, we are left with a dataset with 590 samples.<\/p>\n<\/section>\n

      \n

      3. Data imputation<\/h2>\n

      In some cases, almost all samples used in machine learning models contain missing values, and choosing the first method would significantly lose information.<\/p>\n

      If the data set is small or the number of missing values is considerable, you cannot afford not to use samples with missing values.<\/p>\n

      In such cases, assigning probable values to the missing data is advisable.<\/p>\n

      In this sense, imputation replaces missing data with estimated data.<\/p>\n

      The most common imputation method replaces missing values with the mean value of the corresponding variable.<\/p>\n

      In some situations, the median or the mode is used instead of the mean as the imputation value.<\/p>\n

      dij<\/sub> = vjmean<\/sub>, if dij<\/sub> = NA, i=1,…,p, j=1,…,q.<\/em><\/p>\n

      It is convenient to conduct a preliminary analysis of the data to determine when to replace missing values with the mean or median.<\/p>\n

      If the variable is numerical but has outliers or asymmetric data, it is advisable to replace the missing values with the median.<\/p>\n

      On the contrary, if the variable is numerical and has no outliers, it is advisable to choose the median value of the data.<\/p>\n

      Example: Data imputation<\/h3>\n

      A Portuguese bank institution aims to predict which bank clients will subscribe to a long-term deposit and which will not.<\/p>\n

      The data set used is related to the direct marketing campaigns of a Portuguese bank institution.<\/p>\n

      We have a dataset with 16 variables and 4120 samples.<\/p>\n

      A total of 8350 (12%) missing values and 3591 (87%) missing samples have been recorded.<\/p>\n\n\n\n\n
      Missing values<\/th>\nMissing samples<\/th>\n<\/tr>\n
      8350 (12%)<\/td>\n3591 (87%)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

      We would lose too much information if we eliminated all samples with missing values.<\/p>\n

      To solve the problem, we must conduct a thorough analysis of the data and replace the missing values with the most reasonable estimate.<\/p>\n

      We can decide whether to replace missing values with the mean or the median by calculating the data statistics.<\/p>\n\n\n\n\n\n\n\n\n\n
      Variable<\/th>\nMaximum value<\/th>\nMinimum value<\/th>\nMean<\/th>\nMedian<\/th>\n<\/tr>\n
      age<\/td>\n87<\/td>\n19<\/td>\n41.22<\/td>\n39<\/td>\n<\/tr>\n
      education<\/td>\n3<\/td>\n1<\/td>\n2.15<\/td>\n2<\/td>\n<\/tr>\n
      balance<\/td>\n71182<\/td>\n-3313<\/td>\n1439.81<\/td>\n422<\/td>\n<\/tr>\n
      contact_type<\/td>\n1<\/td>\n0<\/td>\n0.09<\/td>\n0<\/td>\n<\/tr>\n
      last_contact<\/td>\n871<\/td>\n1<\/td>\n223.67<\/td>\n189<\/td>\n<\/tr>\n
      previous_conversion<\/td>\n1<\/td>\n0<\/td>\n0.20<\/td>\n0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

      Taking into account the statistical results and observing no outliers, we can replace the missing values with the mean of the corresponding variable.<\/p>\n<\/section>\n

      \n

      4. Time series data interpolation<\/h2>\n

      This method can be helpful to impute missing values in time series.<\/p>\n

      If we have a dataset with a time series containing missing values, we can give it values by performing interpolation.<\/p>\n

      However, if several consecutive values are missing in a data set column, the interpolation in that period will be less accurate.<\/p>\n

      Example:<\/h3>\n

      An oil company collects production data from oil wells. Analysis of this data is essential to maximize production and detect potential problems.
      \n<\/p>\n

      In the dataset, we have a variable representing the daily volume of oil in cubic meters.
      \n
      \nThis chart shows the missing values in the dataset<\/p>\n

      To treat these missing values, interpolation is carried out.
      \n\"This
      \nThis chart shows the interpolation applied to missing values.<\/p>\n<\/section>\n

      \n

      5. Conclusions<\/h2>\n

      Missing values are entries in the data matrix that are not available.<\/p>\n

      To build a model, you can either remove the samples with missing values or replace them with simple statistics such as the mean or median.<\/p>\n

      More advanced techniques, like nearest-neighbor imputation, can also be applied.<\/p>\n

      Related posts<\/h2>\n<\/section>\n","protected":false},"author":10,"featured_media":1912,"template":"","categories":[],"tags":[36],"class_list":["post-3405","blog","type-blog","status-publish","has-post-thumbnail","hentry","tag-tutorials"],"acf":[],"yoast_head":"\nHandle missing values in machine learning<\/title>\n<meta name=\"description\" content=\"Since the data set is often incomplete, knowing the primary methods for dealing with missing values in machine learning is important.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Handle missing values in machine learning\" \/>\n<meta property=\"og:description\" content=\"Since the data set is often incomplete, knowing the primary methods for dealing with missing values in machine learning is important.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/\" \/>\n<meta property=\"og:site_name\" content=\"Neural Designer\" \/>\n<meta property=\"article:modified_time\" content=\"2025-11-27T14:17:57+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/06\/missing-values-scaled.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"1338\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@NeuralDesigner\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/\",\"url\":\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/\",\"name\":\"Handle missing values in machine learning\",\"isPartOf\":{\"@id\":\"https:\/\/www.neuraldesigner.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/06\/missing-values-scaled.webp\",\"datePublished\":\"2025-10-31T09:59:22+00:00\",\"dateModified\":\"2025-11-27T14:17:57+00:00\",\"description\":\"Since the data set is often incomplete, knowing the primary methods for dealing with missing values in machine learning is important.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#primaryimage\",\"url\":\"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/06\/missing-values-scaled.webp\",\"contentUrl\":\"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/06\/missing-values-scaled.webp\",\"width\":2560,\"height\":1338},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.neuraldesigner.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Blog\",\"item\":\"https:\/\/www.neuraldesigner.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Handle missing values in machine learning\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.neuraldesigner.com\/#website\",\"url\":\"https:\/\/www.neuraldesigner.com\/\",\"name\":\"Neural Designer\",\"description\":\"Explanable AI Platform\",\"publisher\":{\"@id\":\"https:\/\/www.neuraldesigner.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.neuraldesigner.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.neuraldesigner.com\/#organization\",\"name\":\"Neural Designer\",\"url\":\"https:\/\/www.neuraldesigner.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.neuraldesigner.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/05\/logo-neural-1.png\",\"contentUrl\":\"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/05\/logo-neural-1.png\",\"width\":1024,\"height\":223,\"caption\":\"Neural Designer\"},\"image\":{\"@id\":\"https:\/\/www.neuraldesigner.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/NeuralDesigner\",\"https:\/\/es.linkedin.com\/showcase\/neuraldesigner\/\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Handle missing values in machine learning","description":"Since the data set is often incomplete, knowing the primary methods for dealing with missing values in machine learning is important.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/","og_locale":"en_US","og_type":"article","og_title":"Handle missing values in machine learning","og_description":"Since the data set is often incomplete, knowing the primary methods for dealing with missing values in machine learning is important.","og_url":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/","og_site_name":"Neural Designer","article_modified_time":"2025-11-27T14:17:57+00:00","og_image":[{"width":2560,"height":1338,"url":"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/06\/missing-values-scaled.webp","type":"image\/webp"}],"twitter_card":"summary_large_image","twitter_site":"@NeuralDesigner","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/","url":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/","name":"Handle missing values in machine learning","isPartOf":{"@id":"https:\/\/www.neuraldesigner.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#primaryimage"},"image":{"@id":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#primaryimage"},"thumbnailUrl":"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/06\/missing-values-scaled.webp","datePublished":"2025-10-31T09:59:22+00:00","dateModified":"2025-11-27T14:17:57+00:00","description":"Since the data set is often incomplete, knowing the primary methods for dealing with missing values in machine learning is important.","breadcrumb":{"@id":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.neuraldesigner.com\/blog\/missing-values\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#primaryimage","url":"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/06\/missing-values-scaled.webp","contentUrl":"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/06\/missing-values-scaled.webp","width":2560,"height":1338},{"@type":"BreadcrumbList","@id":"https:\/\/www.neuraldesigner.com\/blog\/missing-values\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.neuraldesigner.com\/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https:\/\/www.neuraldesigner.com\/blog\/"},{"@type":"ListItem","position":3,"name":"Handle missing values in machine learning"}]},{"@type":"WebSite","@id":"https:\/\/www.neuraldesigner.com\/#website","url":"https:\/\/www.neuraldesigner.com\/","name":"Neural Designer","description":"Explanable AI Platform","publisher":{"@id":"https:\/\/www.neuraldesigner.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.neuraldesigner.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.neuraldesigner.com\/#organization","name":"Neural Designer","url":"https:\/\/www.neuraldesigner.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.neuraldesigner.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/05\/logo-neural-1.png","contentUrl":"https:\/\/www.neuraldesigner.com\/wp-content\/uploads\/2023\/05\/logo-neural-1.png","width":1024,"height":223,"caption":"Neural Designer"},"image":{"@id":"https:\/\/www.neuraldesigner.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/NeuralDesigner","https:\/\/es.linkedin.com\/showcase\/neuraldesigner\/"]}]}},"_links":{"self":[{"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/blog\/3405","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/blog"}],"about":[{"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/types\/blog"}],"author":[{"embeddable":true,"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/users\/10"}],"version-history":[{"count":1,"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/blog\/3405\/revisions"}],"predecessor-version":[{"id":21402,"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/blog\/3405\/revisions\/21402"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/media\/1912"}],"wp:attachment":[{"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/media?parent=3405"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/categories?post=3405"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.neuraldesigner.com\/api\/wp\/v2\/tags?post=3405"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}