Showing posts with label information. Show all posts
Showing posts with label information. Show all posts

Saturday, December 21, 2019

Characteristics of Computational Systems


1. Information Capacity governed by Shannon's Formula

Shannon's Formula describes the amount of information that can be saved by a system of N states.

S=-\sum _{i}P_{i}\log {P_{i}}
source: wikipedia

Information essentially is an interpretation of system's state. In general for any information system designed by humans, P(i) = P(j) = 1/n, so  formula can be

  S= - Summation(1:n) 1/n log b n 

This will come out to be log n where b is the base and in most cases b=2. So a system with just one state cannot store any information (log 2 1 = 0 ) and a system with two states can store log 2 2 = 1 or one bit of information. 

2. Speed - Switching between states

3. Universality - Which tasks can be solved with a computational system?



Reference - Coursera