undergrad stats/probability is completely fucked--all handwaving and no rigour. how do i learn this stuff properly? i was looking a bit at betancourt's stuff https://betanalpha.github.io/assets/case_studies/modeling_and_inference.html is he any good? what resources should i be using?
>>16519784What is your use case and where do you find you need more rigorous fundamentals? When you say "it's all hand waving and no rigour," can you point to an example of something you find overly hand waving that you don't feel confident about?
>>16519784Start by reading Real Analysis by Royden and Fitzpatrick, then pick up any measure theoretic probability book
Find an actual project worth doing that uses the concepts youre learning at the learning will be 100x easier
>>16519841This. Real analysis, measure theorey, statistical theory. If you aren't overwhelmed with sigma fields by page 2 of the statistical text, you've got the wrong text.
>>16519784See:>>16519303
>>16519784You are showing bayes theorem with is probability, not stats. Clarify what you want to study. Probability is for the most part set theory, calculus w.r.t determining "area" of possibility.Make sure you have a background in multivariable calculus (i.e Calc 3 in America). Important things such as binomial theorem too.Stats is inferences about area. confidence intervals, means, etc. I never learned any of this until I needed to use it so anons are right.
>>16520185>>16519841>>16520126Is learning probability via an applied way not better? For example, there is an entire field of physics that leverages probability/statistics (statistical mechanics).
>>16519844just read this, i recommended op take statistical mechanics. good idea?
>>16519784>All hand waving and no rigorThat's literally everything below graduate level pure math. You're in uni to (hopefully) become a productive member of society and meet your future wife. That's why they make everyone take the same calc 1 course. But you want rigor, and I bet you didn't meet your sweetie freshman year. That's ok. Apply to grad school, and in the meantime, cold approach freshmen so that you can become a creepy, but successful, TA in a few years.
>>16519784go on khan academy dumb ass
>>16520370Learning probability via statistical mechanics is a terrible idea. Statistical mechanics has a lot of built in assumptions that are not in general true (see, for example, Shannon entropy of a discrete random variable vs. Boltzmann entropy). Unless you specifically want to learn statistical mechanics, learning physics to learn probability theory is a stupid idea. Just open a damn probability book and work through that, or at least something more general like machine learning.
>>16520371Statistical mechanics is a good application of probability, but will teach you more about statistical mechanics than probability itself. Statistical mechanics also tends to have a lot of hidden uniformity (and sometimes Gaussianity) baked into it that isn't obvious until you're stuck in a circumstance where those assumptions cause problems.
>>16520432which book?
>>16520590Depends on your level of math education and the level of rigor you want. I'll give you a few, ranked based on audience/difficulty in my opinion (as someone who does statistical signal processing for a living, not a mathematician). Category 1: Calculus based undergraduate probability theory. Doesn't focus strongly on proofs, but should be accessible to anyone with a bit of multivariable calculus and diff-eq background. Forms the basic requirements to approach most master's level statistical inference/machine learning tasks.1) Introduction to Probability and Statistical Inference by Roussas or Introduction to Probability and Statistics for Engineers and Scientists by Sheldon Ross. Both are very applied but are approachable and cover a wide range of topics.2) Introduction to Probability Theory by Bertsekas (a bit more rigorous and starts from set theory fundamentals but covers fewer topics). 3) Probability and Statistics by DeGroot. The "gold standard" for undergraduate calculus based probability theory textbooks. Tries to walk the line between rigor and accessibility/application focus.
>>16520590>>16520651Category 2: Upper level undergraduate/first year graduate.These books generally assume some familiarity with the basics of proof based mathematics and at least some exposure to real analysis and some bits of linear algebra and sometimes a sprinkle of group theory. 1) Probability, Random Variables and Stochastic Processes by Papoulis. The "gold standard" for statistical signal processing oriented probability textbooks. Covers a ton of basic theory as well as meaningful applications to problems in physics and engineering. Split into two parts, the first being a rigorous approach to calculus based probability theory and functions of random variables, and the second being focused on stochastic processes (in particular WSS processes). Ubiquitous in engineering and applied physics master's programs. 2) Probability and Statistical Inference by Mukhopadhay. Less well known than Casella and Berger (to be mentioned next) but personally I prefer this one. Covers probability theory itself in a higher level of detail and is more instructive and focused on being educational (rather than a reference book). 3) Casella and Berger. Technically a statistics textbook but also covers a ton of probability theory in a rigorous fashion. A lot of people love this book. It's not my favorite but I get why people like it. 4) An Introduction to Applied Probability Theory by Bremaud. This book is new, but I love it. Covers a ton of important topics from rigorous set theoretic fundamentals and begins to bridge towards measure theoretic topics towards the end. Much more "pure math" than the three listed before. This is the book that my upcoming first year probability course (in the EE dept.) will be based on.
>>16520657>>16520590Category 2 Honorable Mention: Discrete Probability Models and Methods by Bremaud. If you want to really understand discrete probability theory and discrete Markov processes (or PGMs in general) this book is worth its weight in gold. Doesn't require much more math background than some basic analysis, some adeptness with proof based mathematics, and some calculus/linear algebra. Covers important topics in discrete information theory as well (and if I'm remembering correctly some statistical mechanics, but very much from a mathematicians perspective).Category 3: Measure Theoretic Probability. This is the real stuff. The minimum requirements are serious adeptness with undergraduate/first year graduate real analysis, sometimes some point set topology, and the ability to learn how integrals work. A lot of these books cover measure theory topics, but familiarity with measure in its own context will be very helpful.1) Durrett's Probability: Theory and Examples2) Le'Gall's Measure Theory, Probability and Stochastic Processes (the part of the book covers measure theory directly similarly to a first year course in the topic).3) Bremaud's Probability Theory and Stochastic Processes. Good book, covers a ton of material over a massive number of topics from a rigorous perspective.4) Billingsley Probability and Measure. A lot of people like this one, but I'm not personally a huge fan.5) Kallenberg (don't use this as a first book, but it's the probabilist's Bible for a reason).
>>16520651>as someone who does statistical signal processing for a living, not a mathematiciandang, you guys are my heroes. I always wanted to understand how mobile phone transmissions work. perhaps more than that, i wanted to toy around with it to better my understanding. could you recommend a pathway for this? like the bare minimum and where to go from there based on classes that you took, books, etc. both theory and practice. to save you time, disregard calculus, linear algebra and differential equations
>>16520802You need to have a background in probability theory (especially discrete) and then you should look into Manolakis's digital communications textbook.There's a few basic components of a comms system:1) Encoding (turning the information you want to transmit into blocks of binary encoded data)2) Transmission (turning those blocks of binary encoded data into analog pulses that then are transmitted through the continuous world).3) Channel modeling (how do we model the way the data we've teansmitted will be changed by the medium it passes through)4) Reception (the analog-to-digital process which takes in the received waveforms and turns them back to blocks of binary code)5) Deencoding (finding the best "symbols" or "codes" that fit the one you've received).Cellphone towers mostly use OFDM (as far as I'm aware) and you basically only need linear algebra and Fourier series and then you've got all you need to cover the basics.If you want to try and experiment with this stuff, acoustic and optical communications are pretty easy to setup via Python with simple devices. You could pretty easily set up some viterbi encoding scheme and transmit words over LED pulses or acoustic symbols from one device to another (laptop to laptop, Arduino to laptop etc.)Prof. Peter Willett from UConn has some pretty good lecture notes on his website for a senior undergrad digital communications. If you're interested, take a look at his stuff.
>>16520860Anyone can is a binary logical model the answers are selfreferenced due to reducing factors to known parameters
>>16520870I don't know what that response means (or has to do with what I've said). Shannon's coding theorem tells you that asymptotically it doesn't really matter what kind of code you use from a probability of error perspective, but it does matter in terms of computational efficiency and operation in time-varying environments.
>>16519784You don't. I lost all respect for statistics after witnessing 3 stats profs arguing for nearly an hour over the monty hall problem, that can be solved in a few seconds simply by reviewing all 3 possible combinations in your head. They nearly came to blows, eventually one of them stomped off in a huff and wrote a monte carlo simulation that came back with what - to a highly trained statistics professor - should have been the obvious answer.
>>16520860is this comprehensive enough? https://ee.stanford.edu/~gray/sp.pdfalso, based answer. thanks!
>>16521011Bob gray's books are all pretty good. He's very much an information theorist so that tends to color his perspective on what is important. With that said, information theory was literally developed to solve digital communications problems, so it's not exactly unrelated to your goals if comms is what you want to work on.
>>16520860Is it just me or is signal processing the intersection of computer science, engineering, math and physics? Put plainly, it seems to be the dead center of theory and practice.
>>16520665> Le'Gall's Measure Theory, Probability and Stochastic Processes (the part of the book covers measure theory directly similarly to a first year course in the topic).That looks interesting. Do you know where to download the book for free?
>>16519784I have a BS in EE and had to take a probs and stats class for engineering students. Basically it was the in between that the retards took for other non stem degrees that didn’t use calculus, and the super nerd math shit for a math degree. But my basic assessment after taking that class was that stats was totally made up. It is quite possible I didn’t understand it fully but again my sense of the subject as I was exposed to it were that it was all bullshit. I generally agree with you OP. I’m sure some other turbo nerd will tell me what a midwit I am but I don’t care.
>>16520371I dunno bro there's not that much hardcore statistics in stat mech, usually you establish the ensembles, partition functions etc and then sweep most of the stats shit under the rug.
>>16521136I don't really do a lot of physics (aside from some physics inspired state-space models) but yeah that's a good way to describe it. It's also an incredibly broad term that can cover everything from purely proof based fourier analysis to literal assembly/machine code for synchronizing and demodulating packets from antenna arrays.Personally, the flexibility to cover both rigorous pure math topics and still have marketable skills to remain funded was very appealing to me. There are, as with any engineering field, unfortunately a lot of people who really don't have a passion for the more scientifically/mathematically rigorous sides of things. However, just about any mathematics discipline you can imagine will have some application to signal processing. Galois theory gets used in encoding of mixed continuous/discrete signals. Riemannian geometry gets used in designing sensor arrays (including in dimensions higher than 3d when you consider platform motion). Convex analysis and functional analysis get used in information theoretic approaches to estimation/detection. Hell, even category theory gets used in language/codebook construction for encoding of time-varying and abstracted random signals. >>16521496In this thread, apparently.
>>16521505I'll say the same thing to you as I did to introductory math when they used to ask me "when am I ever going to use this stuff?" The answer being, "if you don't learn it, you definitely won't."Probability theory is about the ratios of areas (generalized to a notion of "measure" later on). Statistics is about trying to get information about "typical" or "average" or "expected" behavior of random variables from deterministic functions of noisy observations. If you take the time to learn and understand those approaches and still decide it isn't for you, you don't need to use them. You could likely live the rest of your life as an office job kind of engineer and never manually calculate a maximum likelihood estimate again. If you do research using empirical data, on the other hand, having some mechanism by which you can extract "typical behavior" from your noisy error-prone observations is quite handy.
>>16520651Was gonna say this entire list looks like someone trained in Electrical Engineering, and somehow I missed you mentioning you doing statistical signal processing.
>>16523983In undergrad I did a double major in EE and applied math. My master's and PhD are EE though. Are my priors/biases too obvious? If I've got an obvious blind spot, what would you recommend as an alternative to my list?
>>16524119No blind spots, just got me smiling because that's the general EE walkthrough of Prob Theory
>>16520919That is crazy
>>16522549Thank you so much. This looks good
>>16526850Wasserman's All of Statistics has a GitHub repo with solutions and Python examples. It's not a terrible idea for an author to have one of they are doing more applied work anyways. It shows people how to apply their content and makes it way more likely people will actually use (and thus buy/cite) your book.
>>16519784Go for Billingsley
solve 60*60 = 90*x without calculating(2*3*10)*(2*3*10) = (3*3*10)*xthus x = 2*2*10 by inspection
>>16519784Sheldon Ross Probability and Statistics helped meout with this. The textbook can be found online
>>16519841I got that book, it's pretty good
>>16519784Wackerly, Degroot and Schervish for rigor.
