Adieu的大杂烩

Recently I got some free time to have some taste of Go. And the first thing I wanted to do was to setup the Go development environment on my Mac. Actually it turned out to be quite easy. Here is what I did:

Install Go

The most convenient way of installing Go is through homebrew. One simple command:

$ brew install go

And you're ready to go.

The $GOPATH magic

Before reading any documentation from Go, I was a little bit curious about how a static type language like Go could manage its dependencies and packages. C didn't do very well in this area and that's one reason I'm always trying to avoid writing C code directly.

I think the designer of Go share the same feeling with me and they made installing 3rd party packages really easy with Go. For example, if you wants to install one package from github, you could just use:

$ go get github.com/user/project

And Go support BitBucket, Google Code as well.

Actually you don't really have to install packages manually, you could simply import them in your code by:

import "github.com/user/project"

And when you run your program, Go will automatically install all 3rd party packages for you. Nice, isn't it?

Another good thing I find is that Go ask you to write your code in packages and you have to put package declaration infront of your source code, like:

package name

That's what I like as a Python developer. I found package based development made my life much easier to write high quality and testable code.

The whole package management experience is a little bit too magic for me. I'd like to know how things work internally. After reading some documents and doing some experiment, I found the magic, the $GOPATH environment variable.

The $GOPATH environment variable holds a directory tells Go where all package source code lives and where to install 3rd party packages. And it holds all compiles libraries too. It's like the standard directory structure to organize your Go project.

Since you could point $GOPATH to different directories, you could easily switch your project environment when you work on different projects.

Virtualenv based Go project environment

Sure, I could add $GOPATH to my .zshrc but I just wanted more. I found virtualenv very useful with Python developmen and I wanted to use it with Go. And virtualenvwrapper I use to manange all my virtual environments just made it easier to use.

First, create the Go virtual environment by:

$ mkvirtualenv go

Then edit the postactivate file in the virtual environment bin directory. Add in:

export GOPATH=$VIRTUAL_ENV

Add edit postdeactivate file. Add in:

unset GOPATH

That's it. When I wanted to work on Go, I just use workon go to switch to the virtual environment and the $GOPATH will automatically set to the virtual environment. The pkg folder will hold all compiles libraries. The src folder will contain all the source code and that's where I should put my source code.

For instance I wanted to write a library called mylib and a application called myapp, and myapp will use mylib as a 3rd party package, I'll make two folders called mylib and myapp in src folder and write code in corresponding folders. When I use import mylib in myapp, Go will link both of them together.

Of course I could create more virtual environments for different Go projects and use workon projectname to switch from them. But as I'm just doing some simple stuff, one single Go environment is already enough for me.

One thing to add, since mylib and myapp are in different folders, I could easily setup git to track each of them. That's much convenient than a single of directory holds everything. I find this writing code in different packages and linking them all together approach really made the code easy to read, maintain and reuse when I use Python. And I'm quite satisfied with what Go provides.

Now it's time to write some Go code.

Comments

周末有时间,打算给 xunlei 实现下载时的Progress Bar,以便用户在远程服务器上下载时可以随时了解当前下载的进度以及下载速度。

Progress Bar的实现方法大致可以分为两种:

  1. 显式更新
  2. 同步更新

显式更新的方法很容易理解,即在当前任务进行的过程中,每隔一段时间更新一次Progress Bar的进度。对于下载任务来说,可以设计为每下载多少个字节,即更新一次进度。显式更新的方法有两个弊端,一是需要让已有任务主动调用,就涉及到如何让一个原本连续执行的任务停下来的问题,特别是对底层函数的调用,不能直接通过修改源码的方式实现。二是当更新Progress Bar进度时,其实是停止了任务的执行,对任务的执行效率有一定影响。

同步更新的方法应当是更加理想的选择。即在任务执行的过程当中,同时更新任务进度。但Python的Thread却不是那么给力。由于GIL的影响造成的性能损失不容忽略。

还好有最近很火的 Coroutine 存在,使用Coroutine可以很容易实现Thread的效果,速度还刷刷的快。真乃居家旅行杀人越货之良伴啊。

这里简单贴一点代码片段出来,更详细的更新见 这个commit

download = gevent.spawn(self.download, url=url, filename=filename)
update_progress = gevent.spawn(display_progress_bar, filename=filename, size=size)
download.link(update_progress)
gevent.joinall([download, update_progress])
def display_progress_bar(filename, size):
    """Display progress bar while downloading"""
    from gevent.greenlet import LinkedExited
    width = 32
    last_size = 0
    try:
        while True:
            if os.path.isfile(filename):
                current_size = os.path.getsize(filename)
                percentage = current_size*100/size
                current_width = width*percentage/100
                sys.stderr.write('% 3d%% [%s%s] %s/s    \r' % (percentage, '#'*current_width, ' '*(width-current_width), filesizeformat(current_size - last_size)))
                last_size = current_size
            time.sleep(1)
    except LinkedExited:
        sys.stderr.write('100%% [%s]\n' % ('#'*width))

简单来说,就是先用gevent生成download和update_progress这两个greenlet。download负责下载,update_progress负责更新Progress Bar。然后告诉download,让它在执行完毕时通知update_progress。最后让这两个greenlet同时执行。display_progress_bar函数会每秒更新一次当前进度,直到收到download执行完成的通知。通知是以LinkedExited异常的形式来传递的。

从这个例子可以看出,在许多Python需要同步执行的场合,使用Coroutine都可以更加简洁高效的完成。Python程序员应当把Coroutine纳入到自己的弹药库储备中来。

Comments

Blog系统使用的是 allbuttonspressed 。这是一款运行在Google Appengine上的开源的简易CMS系统。之前根据自己的需要修改了部分源代码,我部署的版本则停留在了去年5月份左右。这两天花了一些时间将Blog系统更新到最新的版本。

在更新的过程中,发现merge最新的代码产生了不少conflict。为了将来merge的时候变得更加轻松,我修改了部分自定义代码,尽量采取注入式的修改,而不直接修改源代码。

现在,更新后的版本已经上线。从前台几乎看不出区别,后台底层还是有蛮大区别的。

Comments

之前 添加了当博客有新文章发布的时候会自动发送更新到Twitter的功能。由于偷懒,发送的网址并没有缩短,导致今天出现了标题过长时超过Twitter字数限制的问题。

花了一些时间,使用 django-shorturls 为博客添加了简单的网址缩短功能,之后就不用害怕标题太长导致Twitter无法更新的问题了,并且发送到Twitter的更新也更加简洁。

其实,这是一个测试 :)

Comments

前两天想研究一下BlueBox这个新的FreeSwitch网页管理客户端系统,花了一点时间部署了一套。期间遇到了Nginx环境和Kohana框架兼容的问题,以下是遇到的问题以及解决的方案。

背景知识:

  • Nginx :一款高效的Http服务器
  • Kohana :一款PHP框架
  • BlueBox : 使用Kohana开发的一款FreeSwitch网页管理客户端系统,前身是 FreePBX v3

问题及解决方案:

BlueBox的安装参见其相关文档,这里就不重复了。在安装流程中,进行到访问 http://YOUR_WEB_SERVER/bluebox/ 这一步进行初始化时报错。系统会将页面转向到 http://YOUR_WEB_SERVER/bluebox/index.php/installer 这个页面,但是Nginx提示404页面无法找到,无法继续。

经过一番研究之后发现Kohana框架对Apache的支持比较好,在bluebox根目录下也有.htaccess 这个Apache的配置文件。BlueBox的安装文档也是基于Apache环境来编写的。从出错提示来看,应该是直接在index.php后面跟/installer导致Nginx将index.php/installer整体当成了脚本文件名在目录中寻找,最终无法找到,返回404错误。而Apache似乎会将index.php/installer打断,定位到index.php脚本,所以安装流程得以顺利进行。

找到问题之后首先尝试让BlueBox使用Nginx能够解析的url形式,由于并没有Kohana框架的使用经验,简单查看配置文件及Kohana的代码后没有发现解决问题的办法。一番Google以后,找到了 这篇文档 。按照文档中的配置写法,修改了VirtualHost配置,使得Nginx能够正常兼容Kohana框架,问题得到解决。以下是我的配置文件,仅供参考:

server {
    listen       80;
    server_name  YOUR_DOMAIN;
    index        index.html index.htm index.php;
    root         /DOCUMENT_ROOT;

  location ~ .*\.(php|php5)?$ {
      fastcgi_pass  127.0.0.1:9000;
      fastcgi_index index.php;
      include fcgi.conf;
    }

  location /bluebox/ {
    rewrite index.php(.+)$ /bluebox/index.php?kohana_uri=$1 last;
  }
}

其中php-fpm的部分与正常的在Nginx中部署php程序的配置一致,不需要多说,主要是在后面的rewrite rule中处理了index.php与/installer分离的问题。最近多次使用了nginx的rewrite以及try_files,越发觉得nginx在这方面的强大。

当Kohana与Nginx的兼容问题得到解决以后,剩下的安装流程就一切顺利。只是进入BlueBox之后一看,觉得功能还是稍显简单,估计需要再经过一定时间的开发之后,才能真正让人用的顺手。在那之前,我还是继续使用直接修改配置文件的方法来进行FreeSwitch的配置吧。

Comments