Introduction to Scripting

Size: px

Start display at page:

Download "Introduction to Scripting"

Ira Potter
5 years ago
Views:

1 Introduction to Scripting Francesco Vespignani DiSCoF Università degli Studi di Trento. November 13, 2009

2 Today Course Introduction What is Scripting? What is Matlab Basic Programming Concepts Practice

3 This course It is the first module of the Computational and statistical methods for data modelling and analysis course. The course in intended for people that is novice in programming. Why do we offer this course? Your goal: to acquire basic programming skills. Method: learning by doing (half of the lessons will be practice). How I decided to implement the task: using Matlab. Why this is not called a Matlab course?

4 Method Half lesson theoretical, half lesson practical (highly interactive) Do not be happy of the first solution, try to improve the code, explore different ways to solve the same problem Limited (but necessary) homework Evaluation: practical exam, solve a specific problem (very similar to the ones solved during the course) Timetable: Cimec calendar of PC lab google Every Thursday 2pm until December 17 (Exam Janyary 7th?). Slides: Textbooks: Matlab help and demos. Mastering Matlab.

5 What (I hope) you ll learn basic programming concepts (datatypes, input-output, flow-control) basic analysis (how to split a problem into smaller ones) how to (re-)use software, to glue software tools some Matlab, some L A TEX Key software tools we suggest: Matlab and R.

6 A possible program 1. What is scripting, Introduction to Matlab and simple algorithms 2. Strings processing (filenames) 3. Text and binary input-output 4. Array and Matrix (images) 5. Event driven programming (GUI) 6. Computer Graphics: 2d, 3d and animations 7. Advanced text processing: regexp 8. EEGLAB, Psychtoolbox, SPM

7 What is programming, scripting?

8 Software and Science: software as encoded methodology The software you use shapes the way you think? Not really but maybe it may induce a cognitive style in problem solving. E.g. the difference between declarative and imperative programming. In some cases a software product is the way we communicate our know-how (technical, but not only!). E.g. the use of mixed models with R. Writing a code and sharing it with the scientific community is a way to communicate science (at least as important as paper writing). The awareness of these facts is still poor in the scientific community (mainly in older people).

9 Why using Software in Science? We have lots of data, difficult to handle by hands. Get profits from an encoded methodology, being aware of this (choosing the right one). Do not re-invent the wheel, probably somebody else in the world already faced our problem or a similar one. Automation: be faster (only sometimes!) and (try to!) prevent ourself from errors (humans are error prone).

10 Does automation helps in reducing errors? Automation can reduce unpredictable random human errors. However programming errors automatically introduce systematic errors: To avoid errors the rules are: Test the code with different inputs. Uniqueness of the code (try to develop a stable code even if it costs time). Uniqueness of the data (do not duplicate the data on different formats or versions). Take the time is needed to do the job.

11 When to automatize some procedures? Evaluate costs and benefits: usually time sparing is not the goal think about encoding a procedure only if it is somewhat generalizable and it is likely that you (or others!) will reuse it take the time that is needed for learning, development and testing before using in production you encode a method and gain a better understanding of the operations involved programming gives satisfaction (when the program works) the skills you acquire may be useful for your career development you may become famous

12 Which software platform to learn/use? Before using you must learn. A computer language it is easy to learn but it takes time Evaluate carefully the choices, and seek for the (next!) standard. Choose cross-platform tools! When you choose a language you ll be locked-in (you ll try to do everything with it) Consider which is the best (more powerful) instrument Consider which are the standard tools in your discipline Consider which is the diffusion of the tool (even outside science) Don t mind too much: languages and OS are long-living (fortran, cobol) but new standards always try to emerge To understand economic concepts of standards and lock-in within information products see Shapiro and Varian (1998) Information rules Harvard Business Press.

13 De-Facto and Commercial standards Text processing: Microsoft Office, L A TEX Image manipulation: Adobe Photoshop, gimp Technical drawing: Autocad Symbolic Mathematics: Wolfram Mathematica Discrete Mathematics: Mathworks Matlab Measurement and automation: NI Labview and... many more, e.g. specialistic tools such as Praat

14 Open-source, freeware Ethic, ideology... You will not locked in by proprietary file formats.doc.dwg... It is easier to share your scripts with others Many people can look after bugs and errors You have the possibility to understand how it works For science it should be mandatory

15 What is it a script? A program, i.e. a sequence of definitions and instructions. This program is usually interpreted on the fly by an interpreter (another program). Different kind of scripts are: sequences of OS (Operative System) instructions programs in a proper interpreted language sequences of operations embedded as a part of an application. Interpreted languages are opposed to compiled languages. You can read : Ousterhout, J. K. (1998) Scripting: Higher Level Programming for the 21st Century.

16 Scripting versus System Programming Languages System programming languages: designed for building data structures and algorithms from scratch, starting from the more primitive computer elements (words of memory i.e. groups of 0 and 1). Scripting programming languages: designed for gluing, they assume the existence of a set of powerful components and are primarily intended for connecting components together.

17 Layered Architecture High level: Scripting, typically OS independent, re-use of lower levels components. Middle level: System Programming Language (as C) are typically OS dependent, provides functions, implement them for a specific architecture. Lower level: Assembly, machine/os dependent, implements the machine-level code, drivers to specific peripherals etc. Compilers produce assembly code automatically, starting from system language codes. The layered architecture works if and only if the interface is relatively stable. This allows for the independent development within each single layer.

18 System Programming Languages System programming was introduced as an alternative to assembly languages. In assembly virtually any aspect of the machine is reflected in the program (machine specific code). System programming languages were developed between 1950 and 1960 (Lisp, Fortran, Agol) and evolved mainly from Agol to modern languages (Pascal, C, C++, Java). They almost completely replaced assembly for the development of large scale applications.

19 Main differences System programming languages forces you to decide in advance the structure of data that you will use. Scripting programming languages usually allow to underspecify data types and amount of data that will be used (dynamic and automatic memory allocation). Scripting programming languages (interpreted languages) allow to create, test and execute code on the fly, development is much slower with compiled languages. The advantages of scripting are mainly in the speed of development of the program but is payed by more dangers, in terms of unexpected behaviors at runtime, and by lower performances (more run-time checking, dynamic allocation of memory).

20 Many kinds of Scripts Almost any application (EPrime, SPSS, Brain Vision Analyzer, Praat) has its own scripting language to automatize mouse and button operations, sometimes a subset of a proper scripting language, sometimes a specific in-built language with limited capabilities. Operative system languages. Unix shells: sh, csh, bash... MS-DOS (almost defunct!) Interpreted languages: VBasic Perl Python Tcl-tk php

21 What is Matlab?

22 Matlab MATLAB is a high-level language and interactive environment that enables you to perform computationally intensive tasks faster than with traditional programming languages such as C, C++, and Fortran. (mathworks says) It s a commercial (mathworks) product, crossplattform (Win, linux, MacOs), available in many labs all over the world (a commercial standard). It s a (scripting) language with a built-in shell, editor and a complex GUI. It s a de-facto standard in engineering and experimental physical sciences. Matlab product grid encompasses a number of toolbox for specific purposes (bought independently, with separate licenses). It has advantages of fast development of scripting interpreted language and uses efficient mathematical library for numerical computations.

23 Matlab in cognitive (neuro)science There are many useful tools developed by scientist for scientists (freeware, once you have a matlab licence) Examples are. Psychtoolbox SPM EEGLAB Beside to these professionally developed tools you can find out on the web scripts and functions that solve specific problems (pay attention they work correctly!).

24 How to learn Matlab Use the help and the mathworks web site! Let s see some demo videos Getting Started with MATLAB. Let s see some demo videos Working in The Development Environment. All other videos are at See also the help and doc commands. The environment is very useful, try however to do as more as possible by typing in the command window (if you want to learn scripting and not another excel!!!).

25 Basic Programming Concepts

26 Approaching a new language When we approach a new scripting language we need to explore a number of basic features: The basic syntax (typographical conventions, e.g. spaces may have a meaning!) How to interact with the program (invocation, scripts) How to interact with the filesystem and with the OS How data can be stored and organized (variables) How to read and write (input-output) how to control the flow (if... then) how to repeat thinks: cycles, functions and procedures

27 Variables Variables are memory slots where to store data. Depending on the specific language there may be different type of variables. What to know about variables: allowed names basic types structures of variables (arrays, matrixes, cell-arrays...) allocation and storage class visibility

28 Variable assignment in Matlab The = operator assigns the value val to a variable var: var = value Variable type is determined by matlab (implicit typing): a = 1; a = ciccio ; a = true; Look at the workspace (or use whos). Dynamic allocation of arrays: a = 1:5; a(6) = 1000; Remember to use different variables names and initialize them to a value. If you know in advance the size of an array you want to use initialization can reduce processing time.

29 Many ways to perform the same stuff Example with array construction: a = [ ]; a = [1, 2, 3, 4, 5]; a = 1:5; a = 1:1:5; a = linspace(1,5,5);

30 Data Types help datatypes More important datatypes for today: boolean (true=1, false=0) numeric: double (the default) see NaN and Inf values single (see realmax, realmin, eps functions) int32 (see intmax, intmin, eps functions) uint32 (try 4-10 in unint32) Pay attention that discrete mathematics may give counterintuitive results:

31 Operators help ops In particular look and practice with logic operators (notice the difference between single and double =). Operators are applied in a specific order (not left-to-right). See the operator precedence rules on the html documentation. In case you are in doubt use round parentheses to override the natural order. Compare (and try to understand) the following: a = 1:10; b = a.* 2 b = 1:10.* 2 b = (1:10).* 2 Which operator has the precedence? : or.?

32 Flow Control help if help for help while Flow control is of two types: conditional execution (if-else, try-catch) and looping (for, while). While repeats a sequence of instructions conditionally to a boolean value. For repeats it by incrementing an index variable. The break statement exits from a loop. The loops can be nested one into the other and the number of nesting is a measure of the algorithm complexity. It s better that the program exits at some point from the loop (in case try Ctrl-C).

33 Counting: four examples for i=1:10 disp(i) end i=0; while (i<10) i =i+1; disp(i) end i=1; while (i<=10) disp(i) i =i+1; end i=0; while (true) i=i+1; disp(i) if i>=10 break end end

34 While Usually For and While are interchangeable. In some cases while is much better... E.g. let s empirically find the eps value: num=0;eps=1; while (1+EPS)>1 EPS=EPS/2; num = num + 1; end EPS=EPS*2

35 Scripts and functions Once you ve developed and tested a script or a piece of it (that works), save it in a text file with.m extension (or use the matlab command edit or the environment facilities we have seen in the videos). It becomes a new command of matlab, if placed within the matlab search path. If the script is embedded within a function statement your new command may accept variables and return (multiple) values. Variables declared inside functions are not visible in the main workspace (there is a separate namespace and encapsulation that prevent unintentional cross-effect). Much more on this one of the next lessons!!

36 Practice

37 Today practice (homework) Simple algorithms to construct series. a n = f (n, α) α = k n = 1, 2, 3,... power series (e.g. 2 n ) perfect squares integer (4, 9, 16, 25...) fibonacci sequence (1, 1, 2, 3, 5, 8, 13...) prime numbers (1, 2, 3, 5, 7, 11, 13,...) perfect numbers (6, 28, ) factorials (1, 2, 6, 24...).. and other things (maybe next lesson) pitagoric terns (a 2 + b 2 = c 2 ) sort an array

38 Today practice (homework) For each of the problems try to solve it and to see whether there are different ways to solve it. First analyze the problem (make a plan) and then a develop a strategy. Solve a simple case and then try to generalize it by substituting constants with variables (that may become functions arguments). First display the results (and intermediate calculations if needed) on the terminal (command window), then try to store the results for future uses in an array. Test it works with different variables. Find out the limits.

39 Power series 2, 4, 8, 16, with and without using the power function. with and without a loop. using vectors scalar multiplication. make a script and a function for this Perfect squares sequence is similar (4, 9, 16, 25,...).

40 fibonacci sequence (1, 1, 2, 3, 5, 8, 13,...) Definition: a n = a n 1 + a n 2 a 0 = a 1 = 1 n = 1, 2, 3,... If you want to exchange the value of two variables you need a third temporary variable: tmp=a; a=b; b=tmp; Try also a wrong one: tmp=a; b=a; a=tmp;

41 prime numbers 1, 2, 3, 5, 7, Definition: An integer number that can be divided only by 1 and by itself. See the mathematical functions mod, rem. If you find different ways to perform the task to compare computational efficiency try to use the tic, toc commands. Try to see whether there are differences using different datatypes (single, double, integers...).

42 perfect numbers 6 = Definition: A number that is equal to the sum of its proper divisors (they are few).

43 factorial n! = n (n 1) (n 2)... 1 Definition: The product of all the integers smaller of a given integer. This is a function that grows very fast. Test the limits of int32. Try to plot them in a linear of a logarithmic scale (they grow really fast) using the plot function.

Computers and programming languages introduction

Computers and programming languages introduction Eugeniy E. Mikhailov The College of William & Mary Lecture 01 Eugeniy Mikhailov (W&M) Practical Computing Lecture 01 1 / 19 Class goals and structure Primary