The traffic was usually light in my commuting routes this morning so I did get some time for a short excursion before getting into my office at work.  After passing San Mateo bridge, I tried to take some quick landscape photos for fun.  What I always wanted to do was trying out taking some HDR photos and the short excursion was a great opportunity for me to try it out.  The following few photos I put in Flickr are the results from my quick HDR experiment.  Not too bad, although I need to some post-processing.  The color might not look so natural, but anything in modern digital photograph is done with some sort of processing anyway.  The most important thing is probably to have fun to take and develop the pictures anyway.   And, I feel good to have a chance to stop by and observe beautiful thing that has been ignored most of time in my commute route.

http://had.co.nz/plyr/plyr-intro-090510.pdf for data manipulation

http://www.stats.ox.ac.uk/~ruth/RCourse/Rcourse3.pdf

http://cran.r-project.org/doc/contrib/usingR.pdf for R basics

http://www.ats.ucla.edu/stat/r/dae/default.htm with annotated outputs in R

http://cran.r-project.org/doc/contrib/Rossiter-RIntro-ITC.pdf tutorial with info on plots

http://www.statmethods.net/stats/regression.html

http://www.rmetrics.org/ provides an Open Source framework for Financial Analysis.

http://www.econ.uiuc.edu/~econ472/e-Tutorial.html has lecture notes with R code

A brief guide to R and Economics http://people.su.se/~ma/R_intro/R_intro.pdf

http://www.stat.pitt.edu/stoffer/tsa2/index.html has a good beginner’s tutorial for Time Series

http://www.quantmod.com/ provides a great analysis and visualization framework for quantitative trading

http://www.wise.xmu.edu.cn/2007summerworkshop/download/Advanced%20Topics%20in%20Time%20Series%20Econometrics%20Using%20R1_ZongwuCAI.pdf advanced time series in R

Interesting time series packages in R http://robjhyndman.com/software

A Data Mining tool in R http://rattle.togaware.com/

An online e-book for Data Mining with R http://www.liaad.up.pt/~ltorgo/DataMiningWithR/

Advanced Statistics using R http://www.statmethods.net/advstats/index.html

Guide to Credit Scoring using R http://cran.r-project.org/doc/contrib/Sharma-CreditScoring.pdf

http://addictedtor.free.fr/graphiques/ is a graph gallery of R plots and charts with supporting code

A tutorial for Lattice http://osiris.sunderland.ac.uk/~cs0her/Statistics/UsingLatticeGraphicsInR.htm

Ggplot R graphics http://had.co.nz/ggplot2/

Ggplot Vs Lattice @ http://had.co.nz/ggplot/vs-lattice.html

Multiple tutorials for using ggplot2 and Lattice http://learnr.wordpress.com/tag/ggplot2/

Introduction to the Text Mining package in R http://cran.r-project.org/web/packages/tm/vignettes/tm.pdf

Social Network Analysis http://www.r-project.org/conferences/useR-2008/slides/Bojanowski.pdf

Web Scraping in R http://www.programmingr.com/content/webscraping-using-readlines-and-rcurl

http://learnr.wordpress.com/2009/10/06/export-data-frames-to-multi-worksheet-excel-file/ to embed R data frames in Excel via multiple approaches.

http://www.statconn.com/ provides a tool to make R usable from Excel

Connect to MySQL from R http://erikvold.com/blog/index.cfm/2008/8/20/how-to-connect-to-mysql-with-r-in-wndows-using-rmysql

http://www.statmethods.net/input/importingdata.html provides info about pulling data from SAS, STATA, SPSS, etc.

Thematic Maps with R http://stackoverflow.com/questions/1260965/developing-geographic-thematic-maps-with-r

http://smartdatacollective.com/Home/22052 for geographic maps in R

Google Charts with R http://www.iq.harvard.edu/blog/sss/archives/2008/04/google_charts_f_1.shtml

Intro to using RGoogleMaps @ http://cran.r-project.org/web/packages/RgoogleMaps/vignettes/RgoogleMaps-intro.pdf

http://www.stat.uni-muenchen.de/~leisch/Sweave/

R2HTML http://www.feferraz.net/en/P/R2HTML

Poor Man GUI for R http://wiener.math.csi.cuny.edu/pmg/

R Commander is a robust GUI for R http://socserv.mcmaster.ca/jfox/Misc/Rcmdr/installation-notes.html

JGR is a Java-based GUI for R http://jgr.markushelbig.org/Screenshots.html

Tinn-R makes for a good R editor http://www.sciviews.org/Tinn-R/

An Eclipse plugin for R @ http://www.walware.de/goto/statet

Instructions to install StatET in Eclipse http://www.splusbook.com/Rintro/R_Eclipse_StatET.pdf

Komodo Edit R editor http://www.sciviews.org/SciViews-K/index.html

http://www.omegahat.org/ has a very interesting list of packages that is seriously worth a look

Commercial versions of R @ http://www.revolution-computing.com/

A very informative blog http://blog.revolution-computing.com/

Red R for R tasks http://code.google.com/p/r-orange/

KNIME for R http://www.knime.org/introduction/screenshots and is worth a serious look.

—

Comparing to Perl, Python has a quite lagged adoption as the scripting language of choice in the field of bioinformatics, although it is getting some moment recently.   If you read job descriptions for bioinformatics engineer or scientist positions a few year back, you barely saw Python mentioned, even as a “nice to have optional skill”.  One of the reasons is probably lacking of good introductory level bioinformatics books in Python so there are, in general, less people thinking Python as a good choice for bioinformatics.   The book “Beginning Perl for Bioinformatics” from O Reilly was published in 2001.  Almost one decade later, we finally get the book “Bioinformatics Programming Using Python” from Mitchell Model to fill the gap.

When I first skimmed the book “Bioinformatics Programming Using Python”, I got the impression that this book was more like “learning python using bioinformatics as examples” and felt a little bit disappointed as I was hoping for more advanced content.  However, once I went through the book, reading the preface and everything else chapter by chapter, I understood the main target audiences that author had in mind and I thought the author did a great job in fulfilling the main purpose.

In modern biological research, scientists can easily generate large amount of data where Excel spreadsheets that most bench scientists use to process limiting amount of data is no longer an option.  I personally believe that the new generation of biologists will have to learn how to process and manage large amount inhomogeneous data to make new discovery out of it.  This requires general computational skill beyond just knowing how to use some special purpose applications that some software vendor can provide.  The book gives good introduction about practical computational skills using Python to process bioinformatics data.  The book is very well organized for a newbie who just wants to start to process the raw data their own and get into a process of learning-by-doing to become a Python programmer.

The book starts with an introduction on the primitive data types in Python and moves toward the flow controls and collection data type with emphasis on, not surprisingly, string processing and file parsing, two of most common tasks in bioinformatics. Then, the author introduces the object-oriented programming in Python. I think a beginner will also like those code templates for different patterns of data processing task in Chapter 4.  They summarize the usual flow structure for common tasks very well.

After giving the basic concept of programming with Python, the author focuses on other utilities which are very useful for day-to-day work for gathering, extracting, and processing data from different data sources. For example, the author discusses about how to explore and organize files with Python in the OS level, using regular expression for extracting complicated text data file, XML processing, web programming for fetching online biological data and sharing data with a simple web server, and, of course, how to program Python to interact with a database. The deep knowledge of all of these topics might deserve their own books. The author does a good job to cover all these topics in a concise way. This will help people to know what can be done very easily with Python and, if they want, to learn any of those topic more from other resources.  The final touch of the book is on structured graphics. This is very wise choice since the destiny of most of bioinformatics data is very likely to be some graphs used in presentations and for publishing.  Again, there are many other Python packages can help scientists to generate nice graph, but the author focuses on one or two of them to show the readers how to do general some graphs with them and the reader might be able to learn something else from there.

One thing I hope the author can also cover, at least at a beginner level, is the numerical and statistical aspect in bioinformatics computing with Python.  For example, Numpy or Scipy are very useful for processing large amount of data, generating statistics and evaluating significance of the results.  They are very useful especially for processing large amount data where the native Python objects are no longer efficient enough.  The numerical computation aspect in bioinformatics is basically lacking in the book.  The other thing that might be desirable for such a book is to show that Python is a great tool for prototyping some algorithms in bioinformatics.  This is probably my own personal bias, but I do think it is nice to show some basic bioinformatics algorithm implementations in python. This will help the readers to understand a little bit more about some of the common algorithms used in the field and to get a taste on a little bit more advanced programming.

Overall, I will not hesitate to recommend this book to any one who will like to start to process biological data on their own with Python. Moreover, it can actually serve as a good introductory book to Python regardless the main focus on bioinformatics examples. The book covers most day-to-day basic bioinformatics tasks and shows Python is a great tool for those tasks.  I think a little more advanced topics, especially on basic numerical and statistical computation in the book, will also help the target audiences. Unfortunately, none of that topic is mentioned in the book. That has been said, even if you are an experienced python programmer in bioinformatics, the book’s focus on Python 3 and a lot of useful templates might serve well as a quick reference if you are looking for something you do not have direct experience before.

I have recently set up a system such that when I found an interesting URL, I can cut and paste the URL to a widget served from my own host such that it will fetch the resource from the URL, and save it to a location that corresponds to the location of a WebDAV service in my web host account. Then, I can setup the WebDAV compatible file viewer (currently, I am using Air Share Pro) to view the file in my WebDAV direcotry. Specially, you can also download the files in the WevDAV server locally for future offline reading. This is extreme useful. Once the file is local, you don’t have to worry if you get wifi, EDGE, or 3G signal, and you might also save some battery power since you don’t have to reload the files from the network when you read a document on and off.

The main caveat is that you probably need your own web hosting service or iDisk that supports WebDAV. Beside that, you will need to some simple CGI/AJAX programming and spend some money to buy a WebDAV compatible file viewer like Air Share Pro. I won’t be surprised that one day there will be an iPhone app that implements something like this using a similar idea in a more polished way. Nevertheless, such application is always at Apple’s mercy. Once Apple opens some policy in the OS allowing sharing data and files between application like an real operation system, all workarounds will be totally useless. Wait, actually, the way I set up getting file to iPhone is not limited for submitting URL from Mobile Safari. Any browser can push the resource from an URL to the WebDAV directory, regradless if an desktop OS supports mounting a WebDAV volumn. Not bad as an quick method to get some useful document in the Internet from a desktop system to iPhone.

Recently, I purchased a book to know more about F# and I learn some thing that is called “Y-combinator” in functional programming. The F# code for the “Y-combinator” looks like this,

let rec y f x =
  f (y f) x;;

You can apply the “Y-combinator” to some non-recursive functional function to make it recursive. For example,

let fac f = function
  | 0 -> 1
  | n -> n * f (n-1);;

> y fac 5;;
val it : int = 120

I was curious about how to do similar thing in Python. So, I try the following code snippet:

def Y(f):
  def g(x):
    return f(Y(f))(x)
  return g

def fac(f):
  def g(x):
    return 1 if x == 0 else x * f(x-1)
  return g

With these function definitions, “Y(fac)(10)” will give you the correct result 3628800. On the other hand, it is not really easy for me to understand this code as most of my daily programming tasks are in imperative programming style with some more straight forward functional feature. Beside the mathematical way (described in the wiki page) to understand how this work, one way one can see how this work is to see how the code unroll under python interpreter:

Y(fac)(5) -> fac(Y(fac))(5) -> g(5) inside fac with f = Y(fac)

Then the “g(5)” inside fac returns 5 * Y(fac)(4) . The Y(fac)(4) will return 4 * Y(fac)(3) and so on.

It seems there is a lot interesting thing in pure functional world. Hope I will learn more about them soon.

You can download the phdfs.py, and try it out yourself. I have not tested all the methods, so YMMV.

As a postdoc, you don’t get those benefit to students. You are not considered as a formal employee. You don’t get any benefit and you are paid low in the name of science. I still remember that I felt so absurd when I was told I could not pay my monthly parking fee by automatic deduction from my paycheck, because I was a “temporary worker” in the school I had being working for a few years.

Well, I can not say that my career is not benefit from my postdoc research. But, I can not say I totally enjoy being treated by the school as “temporary worker” for an indefinitely amount of time. One should treat the real “working horses” in the academic research industry a little better. Without these working horses, there will be no “super-star” in research communities. Anyway, there is not much point for me to complain anymore. Industrial R&D can be fun too.

『在「百年老店」裡，58歲的馬英九是春秋鼎盛、如日方中的新星。』

『他領導國民黨仆而復起，號召台灣人民、尤其是青年一代，終結了台獨政權，正是「青年創造時代」的典型。』

『愛因斯坦的「相對論」改造了百年間的科學奧秘，而孔孟之道卻歷經千餘年影響世道人心，連馬克斯信徒也不得不信。』

『鼓勵青年學習馬英九，絕不是搞甚麼「偶像祟拜」，更無意要造一座「新神」，而是就近取譬，用大家都看得見的事實，期勉繼往開來的青年世代，好好鑄造自己、鍛鍊自己，無負「青年創造時代」的期望。』

久居國外，我對馬英九了解不算多，也沒有意見。但看了這文章後，不得不想起那連小學裡作文都要以『解救大陸水深火熱同胞』『以三民主義統一中國』的年代。也許，只是也許，某聖君可以不和獨裁磕頭，完成反共復國的大業。這樣就不用每年去拜拜了。

Disclaimer: 我年幼無知的時候為了考試或是混公假，應該也寫了不少奇聞，不過那可是上世紀的歷史共業呀！

奇文原出處之一

我最先發現奇文的地方