{"id":901,"date":"2010-04-04T18:07:19","date_gmt":"2010-04-05T02:07:19","guid":{"rendered":"http:\/\/obliviousrounding.com\/blog\/?p=901"},"modified":"2026-09-30T09:15:58","modified_gmt":"2026-09-30T17:15:58","slug":"probabilistic-method","status":"publish","type":"page","link":"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/","title":{"rendered":"Probabilistic method"},"content":{"rendered":"<div class=\"latex-file\">\n<div class=\"latex-document\">\n<div class=\"latex-outer-sheet\">\n<div class=\"latex-inner-sheet\">\n<blockquote class=\"quote\"><p><center><em>A brief introduction to the probabilistic method, followed by some basic bounds and a few examples.<\/em><\/center><\/p><\/blockquote>\n<p>Alon, Spencer, and Erd\u0151s (1992) describe the method as follows: <\/p>\n<blockquote class=\"quote\"><p><em> In order to prove the existence of a combinatorial structure with certain properties, we construct an appropriate probability space and show that a randomly chosen element in the space has the desired properties with positive probability.<\/em><span class=\"cite\">[<a href=\"#Alon92Probabilistic\">1<\/a>]<\/span> <\/p><\/blockquote>\n<p>The method has come into wide use since about 1950, even in areas of mathematics that, a-priori, have nothing to do with probability. This is roughly because probability provides a useful high-level conceptual framework for understanding (formalizing, organizing, communicating) many widely applicable counting and averaging arguments. This probabilistic viewpoint is called the <em>probabilistic lens<\/em>. <\/p>\n<p>Alon et al. note that, although <em>in principle<\/em> analysis with the probabilistic method can be replaced by direct arguments, <\/p>\n<blockquote class=\"quote\"><p><em> \u2026in practice, the probability is essential. It would be hopeless to replace the applications of many of the tools [by counting arguments].\u00a0<span class=\"cite\">[<a href=\"#Alon92Probabilistic\">1<\/a>, page 2]<\/span><\/em> <\/p><\/blockquote>\n<p>In Computer Science, the method is used in <a href=\"http:\/\/algnotes.info\/on\/randomized-rounding\">randomized rounding<\/a>, to design approximation algorithms for combinatorial optimization problems.<!--more--><\/p>\n<p><div style=\"margin:0pt;padding:0pt;margin-bottom:12px\"><\/div>\n<\/p>\n<hr \/>\n<div class=\"expandable\" id=\"a0000000002\" tabindex=\"10\">\n<details class=\"collapseomatic_details\">\n<summary title=\"Click for background material\u2026\" class=\"collapseomatic highlight\" tabindex=\"0\">Click for background material\u2026<\/summary>\n<div class=\"collapseomatic_content\">\n<ul class=\"itemize\">\n<li>\n<p>Familiarity with the basic concepts of probability: random experiments, variables, and events. (Reference materials on these topics are listed at the end, below.) <\/p>\n<\/li>\n<\/ul>\n<\/div>\n<\/details>\n<\/div>\n<hr \/>\n<\/p>\n<\/div>\n<\/div>\n<div class=\"latex-outer-sheet\">\n<div class=\"latex-inner-sheet\">\n<div class=\"latex-section\">\n<h1 id=\"a0000000003\">Notes on the probabilistic method<\/h1>\n<div class=\"ccchildpages\"><article class=\"hentry ccchildpage\">\r\n<header class=\"entry-header\">\r\n<h1 class=\"entry-title\">\r\n<a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/basic-bounds\/\" class=\"\">basic bounds<\/a>\r\n<\/h1>\r\n<\/header>\r\n<div class=\"entry-summary\">\r\n<div class=\"latex-file\"><div class=\"latex-document\"> <div class=\"latex-outer-sheet\"><div class=\"latex-inner-sheet\"> <p><blockquote class=\"quote\"><center><em>Existence proofs, the naive union bound, linearity of expectation, Markov bound.<\/em><\/center><\/blockquote>\r\n<!--- a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/basic-bounds\/\" class=\"\">More...<\/a --->\r\n<\/div>\r\n<footer class=\"entry-meta\">\r\n<\/footer>\r\n<\/article><article class=\"hentry ccchildpage\">\r\n<header class=\"entry-header\">\r\n<h1 class=\"entry-title\">\r\n<a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/examples\/\" class=\"\">examples (Max Cut, Tur\u00e1n\u2019s theorem)<\/a>\r\n<\/h1>\r\n<\/header>\r\n<div class=\"entry-summary\">\r\n<div class=\"latex-file\"><div class=\"latex-document\"> <div class=\"latex-outer-sheet\"><div class=\"latex-inner-sheet\"> <p><blockquote class=\"quote\"><center><em>Examples: 2-approximation for Max Cut; Tur\u00e1n\u2019s theorem<\/em><\/center><\/blockquote>\r\n<!--- a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/examples\/\" class=\"\">More...<\/a --->\r\n<\/div>\r\n<footer class=\"entry-meta\">\r\n<\/footer>\r\n<\/article><article class=\"hentry ccchildpage\">\r\n<header class=\"entry-header\">\r\n<h1 class=\"entry-title\">\r\n<a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/method-of-conditional-probabilities\/\" class=\"\">method of conditional probabilities (Max Cut)<\/a>\r\n<\/h1>\r\n<\/header>\r\n<div class=\"entry-summary\">\r\n<div class=\"latex-file\"><div class=\"latex-document\"> <div class=\"latex-outer-sheet\"><div class=\"latex-inner-sheet\"> <p><blockquote class=\"quote\"><center><em>The method of conditional probabilities converts a probabilistic existence proof into a deterministic algorithm.<\/em><\/center><\/blockquote>  The method of conditional probabilities is a systematic method for converting non-constructive probabilistic existence proofs into efficient deterministic algorithms that explicitly construct the desired object.\r\n<!--- a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/method-of-conditional-probabilities\/\" class=\"\">More...<\/a --->\r\n<\/div>\r\n<footer class=\"entry-meta\">\r\n<\/footer>\r\n<\/article><article class=\"hentry ccchildpage\">\r\n<header class=\"entry-header\">\r\n<h1 class=\"entry-title\">\r\n<a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/pessimistic-estimators\/\" class=\"\">pessimistic estimators (Tur\u00e1ns theorem)<\/a>\r\n<\/h1>\r\n<\/header>\r\n<div class=\"entry-summary\">\r\n<div class=\"latex-file\"><div class=\"latex-document\"> <div class=\"latex-outer-sheet\"><div class=\"latex-inner-sheet\"> <p><blockquote class=\"quote\"><center><em>Pessimistic estimators in the method of conditional probabilities.<\/em><\/center><\/blockquote>  In applying the method of conditional probabilities, exact conditional probabilities (or expectations) are sometimes hard to compute. Pessimistic estimators can be used instead. We illustrate the idea by example, using Tur\u00e1n\u2019s theorem.\r\n<!--- a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/pessimistic-estimators\/\" class=\"\">More...<\/a --->\r\n<\/div>\r\n<footer class=\"entry-meta\">\r\n<\/footer>\r\n<\/article><article class=\"hentry ccchildpage\">\r\n<header class=\"entry-header\">\r\n<h1 class=\"entry-title\">\r\n<a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/chernoff\/\" class=\"\">Chernoff bound<\/a>\r\n<\/h1>\r\n<\/header>\r\n<div class=\"entry-summary\">\r\n<div class=\"latex-file\"><div class=\"latex-document\"> <div class=\"latex-outer-sheet\"><div class=\"latex-inner-sheet\"> <p><blockquote class=\"quote\"><center><em>A brief introduction to Chernoff bounds.<\/em><\/center><\/blockquote><p> If you\u2019re already familiar with Chernoff bounds, you may prefer to <a href=\"http:\/\/algnotes.info\/on\/chernoff-proof\">skip directly to the statement and proof of a typical Chernoff bound.<\/a> <\/p><p>Chernoff bounds (a.k.a.\u00a0tail bounds, Hoeffding\/Azuma\/Talagrand inequalities, the method of bounded differences, etc.\u00a0<span class=\"cite\">[<a href=\"#Chernoff52Measure\">1<\/a>, <a href=\"#Hoeffding63Probability\">2<\/a>]<\/span>) are used to bound the probability that some function (typically a sum) of many \u201csmall\u201d random variables falls in the tail of its distribution (far from its expectation).\r\n<!--- a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/chernoff\/\" class=\"\">More...<\/a --->\r\n<\/div>\r\n<footer class=\"entry-meta\">\r\n<\/footer>\r\n<\/article><article class=\"hentry ccchildpage\">\r\n<header class=\"entry-header\">\r\n<h1 class=\"entry-title\">\r\n<a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/chernoff-proof\/\" class=\"\">Chernoff bound proof<\/a>\r\n<\/h1>\r\n<\/header>\r\n<div class=\"entry-summary\">\r\n<div class=\"latex-file\"><div class=\"latex-document\"> <div class=\"latex-outer-sheet\"><div class=\"latex-inner-sheet\"> <p><blockquote class=\"quote\"><center><em>The statement and proof of a typical Chernoff bound.<\/em><\/center><\/blockquote> \u00a0\u00a0\r\n<!--- a href=\"https:\/\/algnotes.info\/on\/background\/probabilistic-method\/chernoff-proof\/\" class=\"\">More...<\/a --->\r\n<\/div>\r\n<footer class=\"entry-meta\">\r\n<\/footer>\r\n<\/article><\/div>\n<div class=\"latex-subsection\">\n<h2 id=\"a0000000004\">Reference materials on basic probability, available online<\/h2>\n<ul class=\"itemize\">\n<li>\n<p><a href=\"https:\/\/www.google.com\/search?q=%22Notes+on+Discrete+Probability%22+trevisan\">Notes on Discrete Probability<\/a> \u2014 by Trevisan <\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/www.google.com\/search?q=%22Introduction+to+Probability%22+Grinstead+Snell+pdf\">Introduction to Probability<\/a> \u2014 by Grinstead and Snell (see Chapters 1, 4, 6) <\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/www.google.com\/search?q=%22Lecture+Notes+on+Probability+Theory+and+Random+Processes%22+Walrand+pdf\">Lecture Notes on Probability Theory and Random Processes<\/a> \u2014 by Walrand (see Chapters 1\u20136) <\/p>\n<\/li>\n<\/ul>\n<\/div>\n<div class=\"latex-subsection\">\n<h2 id=\"a0000000005\">The probabilistic method<\/h2>\n<ul class=\"itemize\">\n<li>\n<p>Textbooks:\u00a0<span class=\"cite\">[<a href=\"#Alon92Probabilistic\">1<\/a>]<\/span>,\u00a0<span class=\"cite\">[<a href=\"#Motwani95\">2<\/a>, ch. 5]<\/span> <\/p>\n<\/li>\n<li>\n<p>Wikipedia: <a href=\"http:\/\/en.wikipedia.org\/wiki\/Probabilistic_method\">The probabilistic method<\/a>; <a href=\"http:\/\/en.wikipedia.org\/wiki\/Probabilistic_proofs_of_non-probabilistic_theorems\">Probabilistic proofs of non-probabilistic theorems<\/a> <\/p>\n<\/li>\n<li>\n<p>Here: <a href=\"http:\/\/algnotes.info\/on\/randomized-rounding\">Randomized rounding<\/a>; <a href=\"http:\/\/algnotes.info\/on\/obliv\">Using randomized rounding to derive greedy and Lagrangian-relaxation algorithms<\/a>. <\/p>\n<\/li>\n<\/ul>\n<\/div>\n<\/div>\n<div>\n<h1 id=\"bibliography\">Bibliography<\/h1>\n<table class=\"bibliography\" cellspacing=\"0\" cellpadding=\"2\">\n<tr>\n<td valign=\"top\">[<a name=\"Alon92Probabilistic\">1<\/a>]<\/td>\n<td><a href=\"http:\/\/scholar.google.com\/scholar?q=author:&quot;N+Alon+&quot;+author:&quot;+J+H++Spencer&quot;+intitle:&quot;+The+Probabilistic+Method&quot;\">N.\u00a0Alon and J.\u00a0H. Spencer. <em>The Probabilistic Method<\/em>. John Wiley and Sons, New York, 1992. <\/a><\/td>\n<\/tr>\n<tr>\n<td valign=\"top\">[<a name=\"Motwani95\">2<\/a>]<\/td>\n<td><a href=\"http:\/\/scholar.google.com\/scholar?q=author:&quot;R+Motwani+&quot;+author:&quot;+P+Raghavan&quot;+intitle:&quot;+Randomized+Algorithms&quot;\">R.\u00a0Motwani and P.\u00a0Raghavan. <em>Randomized Algorithms<\/em>. Cambridge University Press, 1995. <\/a><\/td>\n<\/tr>\n<\/table><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>A brief introduction to the probabilistic method, followed by some basic bounds and a few examples. Alon, Spencer, and Erd\u0151s (1992) describe the method as follows: In order to prove the existence of a combinatorial structure with certain properties, we construct an appropriate probability space and show that a randomly chosen element in the space [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":8676,"menu_order":1,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-901","page","type-page","status-publish","hentry"],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/algnotes.info\/on\/wp-json\/wp\/v2\/pages\/901","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/algnotes.info\/on\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/algnotes.info\/on\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/algnotes.info\/on\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/algnotes.info\/on\/wp-json\/wp\/v2\/comments?post=901"}],"version-history":[{"count":0,"href":"https:\/\/algnotes.info\/on\/wp-json\/wp\/v2\/pages\/901\/revisions"}],"up":[{"embeddable":true,"href":"https:\/\/algnotes.info\/on\/wp-json\/wp\/v2\/pages\/8676"}],"wp:attachment":[{"href":"https:\/\/algnotes.info\/on\/wp-json\/wp\/v2\/media?parent=901"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}