As a math student who is interested in information theory and neural networks, i discovered this gem of a lecture series when i was looking for videos to fall asleep to! In fact i've finished the lectures when i was sleeping :D Now I decided to start it properly and just finished watching this lecture and taking notes. I would love to send David a mail when i finish the course. Thanks for leaving this behind my man, rest in peace.

RIP prof David, you were, and still a great inspiration to us

Fundamental Problem: Reliable communication over unreliable channel Binary Symmetric Channel: 9:35 Disk Drive Problem: 11:30 Redundancy: 23:00 Repetition Code: 24:35 Decoding is inference Inverse Probability: 31:33 Forward Probability: 40:30 Hamming Code: 47:10 (kzfaq.info/get/bejne/m8-odqqiydKrqIU.html%29) Capacity of channel: 58:10 Shannon Noisy Channel Coding Theorem: 58:45 The Weighing Problem 1:00:50

I am not involved in information technology but this lecturer is making a difficult subject like information theory look so easy. You really must see this...

One of the best lecture of " information theory and coding " I have ever seen....love from India 🇮🇳🇮🇳🇮🇳🇮🇳

Dr. Mackay is a great explainer. Anyone interested in machine learning and Bayesian statistics can also read his doctoral thesis.

I was saddened to read that David MacKay passed away earlier this year.

How am I just now discovering this lecture series??! This is awesome!

Thank you for these great lectures your memories will live on ... RIP

Thank you, Thank you, Thank you for sharing these excellent video lectures. Dr Mackay is amazing at teaching complicated topics. These lectures are great supplement to his excellent book on information theory which has so many excellent plots and graphs that enables one to visualize information theory. Information theory comes alive in pictures. Thank you for sharing these.

This may be the longest blackboard I've ever seen

Repetition (redundancy) is dual to variation -- music. Certainty is dual to uncertainty -- the Heisenberg certainty/uncertainty principle. Syntropy (prediction) is dual to increasing entropy -- the 4th law of thermodynamics. Randomness (entropy) is dual to order (predictability) -- "Always two there are" -- Yoda. Teleological physics (syntropy) is dual to non teleological physics. Duality: two sides of the same coin.

what an amazing lecture and great teacher. May you rest in peace. You will not be forgotten.

I don't mind the slowness. With some decoding, my brain is receiving a cleaner signal with that extra redundancy.

It was amazing, Finally I learned Shannon noisy channel theorem: there is exist an encoding and decoding system that could reach to the capacity of channel. so error correcting and detecting course is about to learn these encoding and decoding system. wow Amazing lots of thanks to the teacher

00:30 Information Theory invented by Claude Shannon to solve communication problems. Fundamental problem: Reliable communication over an unreliable channel. e.g. [Voice->(Air)->Ear], [Antenna->(Vacuum)->Mars Rover], [Self->(magnetized film)->Later Self] 04:00 Received signal is approximately equal to the transmitted signal because of added noise. 05:45 What solutions are there for having received and transmitted signal be the same? Either physical solutions or system solutions. 07:30 Source message -> [Encoder] -> Coded transmission -> [Channel (noise introduced)] -> Received message -> [Decoder] -> Best guess at original source 08:45 Encoder is some system that adds redundancy. Decoder makes use of this known system to try to infer both the source message and n.

So nice to watch a video like this that does not have music.

This is a great lecture. The design of case problem is really helpful here. Thanks for the lecture.

Thankyou very much prof David MacKay, Your lecture helps me understand information theory. Btw I live in the country where toilet is not so common

This is great, thank you! The lecture is far more entertaining than just reading the book.

Genius camera work

Thank you for a great lecture, looking forward to follow the rest and study the book!

this really helps. I really wanted to learn information theory. this video series is really easy to understand and awesome.

awesome example of teaching style

These lectures are great. Thanks for sharing.

I looove the lectures! Thank you for the upload!

I like this channel. I'm already teaching this topic for student in Iraq. In Arabic.

This is fantastic I love it.

I would use SUDOKUS for communication. Only few numbers have to be transmitted correctly, and the other numbers/information can be restored by the decoder :-D

Ah! thanks, was looking for a good series... this is just the one. A great cliffhanger at the end to be precise... ^_^

15:52

9:50

Could anyone explain how is the flip pobability is 10^-13 and and 1% failure is 10^-15. Video reference is at 22:10

32:13

Cheering, in these days of advanced educational gizmolgy, to see Professor MacKay making extensive and effective use of a stick of chalk and a polychromatically absorbent surfaced board. {Tyneside, England.]

Thank you professor

awesome! thanks for sharing

3/8 3/8 2/8 cake and siblings That's the most equal way to share In the 3rd cake they are most close to equal 4/82/8 2/8 is far behind equal

J'ai compris pourquoi répéter 3 fois la même information de suite à mes enfants n' était pas forcément efficace :-)

Thanks for these.

I wonder if he was making a joke when he said "Mr. Binomial" or if he actually meant to say "Bernoulli".

I'm new to information theory. At 4:24 why is the received signal is "approximately" and not "equal" to the transmitted signal + noise? like what else is there other than the transmitted signal and the noise? Thanks for sharing such helpful lectures.

Any idea about the final puzzle? My answer is 3. :-)

at 46:50, what is rate? i guess it is (1/no of repetitions) but what does it mean in layman terms

"People need 20 GB drives nowadays" Laughs in Call of Duty

After listening to this lecture, my IQ went up 20 points!

I am curious to know how Shannon would have interpretted the internet as part of his theory... noise perhaps

Helpful lecture

2:00

very great theory

towards 56:00 he uses the term "bit error" and "block error" but doesnt define them properly..

is dis how u mak a plane

I'm a bit curious about when these Lectures were recorded. Was it 2003??

38:19 I am unsure of the 1/2 in the denominator According to my calculations P(r=011) = (1-f)f^2+f(1-f)^2 This is done by the sum rule. The numerator should have 1/2 due to the P(s=1) term My final answer is P(s=1|r=001) = (1-f)/2 Could anybody help me with my concerns?

K

the link of getting free book is not working,can anybody tell from where i can get that free book? Thanks in advance

can someone explain how to solve the homework problems?

I have read the extraordinary book.

🎉

I'm a bit confused. If we're trying to estimate the mean of the Binomial distribution, wouldn't we use the sigma/sqroot(n) formula for the +- bound? In other words, the mean +- the standard deviation of the estimator: 1000 +- 30/sqroot(10000) = 1000 +- 0.3? My statistics is a bit rusty.

Was Mr. Bionomial a joke that fell on dead ears or was it a genuine confusion with Bernoulli?

Information theory

im learning this at school... yet I'am here watching a lecture on KZfaq ... I dont know why ... I should pay attention in my class I guess.

hey is there any pre-requisite of it

Hi, I just wanted to ask what was meant when at 24:20 when he says that "1 is the same as 5, and if there was a 4, there would be a 0."

I'm fairly new to this. What is a "flip" ?

What are the requirements to understand this lecture?

Good lecture except for the highly distracting camera work.

You know he's a jokster cuz at 1:00:13 the slide says his textbook weighs "~35 lbs".

Thanks for the lectures. I am just curious about what happened to lecture 2. It says it was deleted.

Binomial is more related to probability topics. Bernoulli is about hydraulics.

rip david

I was able to achieve f=0.1 for this video with x1.5 speed ...

the fact is the received signal is not identical to sent signal due to corruption and distortion in the signal, in a process , so how much of the the original signal is received ,what would be the measurement in what unit ,,,,,thats why i do drugs ,and dont give a damm !

channel immigration

hehe if you re like me, then you have that exam in one week

Profesor McKay is certainly extremely competent in the subject but this really is the most un-intuitive way of introducing information theory. We are given the answer right from the start and work our way backwards to see that it is an effective system, instead of trying to understand the problem and find the adequate solution. We are never trying to understand the nature of the problem but instead made to test the effectiveness of the solution. Typical of classical academic philosophy. Let's make knowledge as boring and abstruse as possible so the riff raff is kept out of our little club!

This is painfully slow. I'm sure the information is fantastic but you have to watch him write everything he says on the chalkboard. When I got to him telling the class to discuss how much 10% of 10,000 is I couldn't take it anymore. Can anyone suggest a similar lecture that moves more quickly?

Where in the fock am I?

There is no information here and when will it be realized?Never!!!

In the first example, p is equal to f? p should be 1-f = 0.90

13:24

34:43

8:22