[ Content | Sidebar ]

Archives for 九月, 2011

bash拼音补齐靠谱版本

之前玩ubuntu的时候写过一个,功能和代码都较丑,用的拼音库也很烂。正好为项目抓了一个靠谱的拼音库,顺便把这个bash补齐又重新实现了下,Mac上也可以用了~

项目:http://code.google.com/p/bash-pinyin-completion/

==特性==
* 支持拼音首字母匹配和完全匹配
* 支持多音字匹配

安装

ubuntu

支持10.04,10.10,11.04版本从ppa安装,如果之前安装过那个chs-completion先卸载之

sudo add-apt-repository ppa:emptyhua/toolbox
sudo apt-get update
sudo apt-get install bash-pinyin-completion

然后新开一个终端,就可以使用拼音补齐了

Mac OS X

对于mac需要先安装 bash-completion

sudo port install bash-completion

然后编辑~/.bash_profile,把下面的代码贴进去

if [ -f /opt/local/etc/bash_completion ]; then
. /opt/local/etc/bash_completion
fi

下载源码包http://code.google.com/p/bash-pinyin-completion/downloads/list

unzip bash-pinyin-completion-xxx.zip
cd bash-pinyin*
make
sudo make install

然后新开一个终端,就可以使用拼音补齐了

linux中tty设备名的由来

tty是teletypewriter(电传打字机)的简称,电传打字机是一种老式的通信工具,类似于传真,它有一个用于输入的键盘和用于输出接收信息的纸带打印设备。当按下按键后,电传机会把对应的键码编码成电流脉冲发送给接收方。对方收到脉冲后解码,并将对应的字符打印到纸带上。

早期的一些计算机便使用电传机来做输入输出设备。用户通过电传机向计算机输入指令,并通过纸带打印计算机的反馈结果,这便是最原始的command line interface(命令行交互)。

unix系统会为所有设备在/dev目录下生成对应的文件,电传机便被命名为/dev/tty#(#代表数字序号)。虽然现在的计算机早已经不再使用电传机做IO设备,但是tty的命名依然沿用至今。

links:
http://www.linfo.org/teletype.html
http://baike.baidu.com/view/1773688.htm

使用phantomjs抓取ITC和android market的安装统计数据

同事用python抓各大andriod市场的页面分析软件下载量,发现andriod market的数据是通过ajax加载的,而且貌似加密过了,没法直接解析。后来俺发现了phantomjs这个命令行的webkit,在可以在命令行渲染网页,这样不论数据怎么加载,怎么加密,对于标准的浏览器都无能为力了。软件的原理跟以前介绍的一个命令行web截图工具一样,内嵌一个qt4的webkit,然后渲染到xvfb虚拟出的xserver上。不同是这个提供了js的api,用起来方便,可以用来做爬虫,站点监控,服务端截图。

使用方法:
第一步,安装phantomjs
Mac os & windows:
直接下载.dmg或.exe安装包即可:http://code.google.com/p/phantomjs/downloads/list
mac下安装完闭,可执行文件的路径:/Applications/phantomjs.app/Contents/MacOS/phantomjs
ubuntu:

sudo add-apt-repository ppa:jerome-etienne/neoip
sudo apt-get update
sudo apt-get install phantomjs
Copy Code 

centos 5.3:
折腾开始了。。由于phantomjs的linux版本是通过pyqt4实现的,所以装起来比较麻烦
首先我们需要安装qt4.7.而yum默认安装的是4.1

rpm -ivh http://software.freivald.com/centos/software.freivald.com-1.0.0-1.noarch.rpm
yum update fontconfig fontconfig-devel
yum install qt4 qt4-devel
#如果已经安装过qt4则执行 yum update qt4 qt4-devel
Copy Code 

安装Xvfb

yum install xorg-x11-server-Xvfb xorg-x11-server-Xorg xorg-x11-fonts*
Copy Code 

安装python 2.7,自带的为2.4,没法用

wget http://python.org/ftp/python/2.7.2/Python-2.7.2.tar.bz2
tar jxvf Python-2.7.2.tar.bz2
cd Python-2.7.2
./configure --prefix=/opt/python27
make
make install
cd ..
Copy Code 

安装setup tools

wget http://pypi.python.org/packages/source/s/setuptools/setuptools-0.6c11.tar.gz#md5=7df2a529a074f613b509fb44feefe74e
tar zxvf setuptools-0.6c11.tar.gz
cd setuptools-0.6c11
/opt/python27/bin/python setup.py install
cd ..
Copy Code 

安装sip

wget http://www.riverbankcomputing.com/static/Downloads/sip4/sip-4.12.4.tar.gz
tar zxvf sip-4.12.4.tar.gz
cd sip-4.12.4
/opt/python27/bin/python configure.py
make
make install
cd ..
Copy Code 

安装pyqt4

wget http://www.riverbankcomputing.com/static/Downloads/PyQt4/PyQt-x11-gpl-4.8.5.tar.gz
tar zxvf PyQt-x11-gpl-4.8.5.tar.gz
cd PyQt-x11-gpl-4.8.5
/opt/python27/bin/python configure.py -q /usr/lib/qt4/bin/qmake
#对于64位系统
#/opt/python27/bin/python configure.py -q /usr/lib64/qt4/bin/qmake
make 
make install
cd ..
Copy Code 

最后安装pyphantomjs

mkdir pyphantomjs
cd pyphantomjs
wget http://phantomjs.googlecode.com/files/pyphantomjs-1.2.0-source.zip
unzip pyphantomjs-1.2.0-source.zip
/opt/python27/bin/python setup.py install
Copy Code 

折腾到这里,pyphantomjs已经安装到了/opt/python27/bin/pyphantomjs
直接执行/opt/python27/bin/pyphantomjs –help,会发现报错

sip.setapi('QString', 2)
ValueError: API 'QString' has already been set to version 1

解决方法
编辑/opt/python27/bin/pyphantomjs,在开始追加几句

#!/opt/python27/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'PyPhantomJS==1.2.0','console_scripts','pyphantomjs'
#fix start
import sip
sip.setapi('QString', 2)
sip.setapi('QVariant', 2)
#fix end
__requires__ = 'PyPhantomJS==1.2.0'
Copy Code 

第二步:
下载我们用来解析数据的js脚本:android_itc_daliy_report

修改里面的变量:

TIMEOUT = 120;//脚本执行超时时间
ACCOUNT = '';//登陆账号
PASSWORD = '';//密码

第三步:执行抓取脚本
对于Mac os:

#抓取andriod market的安装总量
/Applications/phantomjs.app/Contents/MacOS/phantomjs --load-images=no AndroidMarketDailyReport.js
#抓取ITC的每天的安装量,需要制定日期,而且日期必须是web页面里的日期选择框里有的日期
/Applications/phantomjs.app/Contents/MacOS/phantomjs --load-images=no ITCDailyReport.js 09/06/2011
Copy Code 

对于Centos:

#首先保证Xvfb已经启动
Xvfb :0 -screen 0 1024x768x24 &
#抓取andriod market的安装量
DISPLAY=:0 /opt/python27/bin/pyphantomjs --load-images=no --ignore-ssl-errors=yes AndroidMarketDailyReport.js
#抓取ITC的每天的安装量,需要制定日期,而且日期必须是web页面里的日期选择框里有的日期
DISPLAY=:0 /opt/python27/bin/pyphantomjs --load-images=no --ignore-ssl-errors=yes ITCDailyReport.js 09/06/2011
Copy Code 

获取输出结果,以Mac os为例:

/Applications/phantomjs.app/Contents/MacOS/phantomjs –load-images=no ITCDailyReport.js 09/06/2011|grep REPORT
REPORT: soft_name 0000
REPORT: soft_name 0000

/Applications/phantomjs.app/Contents/MacOS/phantomjs –load-images=no AndroidMarketDailyReport.js |grep REPORT
REPORT: total 0000
REPORT: real 0000

如果没有输出结果,则说明有异常,账号错误,超时,等等。。

key code table

Key Mac Windows Linux Notes
rbKeyUp 126 26 103
rbKeyDown 125 28 108
rbKeyLeft 123 25 105
rbKeyRight 124 27 106
rbKeyBackspace 117 8 14
rbKeyEnter 76 * 28
rbKeyHome 115 36 102
rbKeyEnd 119 35 107
rbKeyPageDown 121 34 109
rbKeyPageUp 116 33 104
rbKeyReturn 36 13 *
rbKeyDelete 51 46 111
rbKeyTab 48 9 15
rbKeySpacebar 49 20 57
rbKeyShift 56 10 *
rbKeyControl 59 11 *
rbKeyMenu 58 18 139 The Alt key
rbKeyPrintScreen * 42 210
rbKeyEscape 53 27 1
rbKeyCapsLock 57 20 58
rbKeyHelp 114 47 138
rbKeyF1 122 112 59
rbKeyF2 120 113 60
rbKeyF3 99 114 61
rbKeyF4 118 115 62
rbKeyF5 96 116 63
rbKeyF6 97 117 64
rbKeyF7 98 118 65
rbKeyF8 100 119 66
rbKeyF9 101 120 67
rbKeyF10 109 121 68
rbKeyF11 103 122 87
rbKeyF12 111 123 88
rbKeyMacFn 63 * *
rbKeyMacOption 58 * *
rbKeyMacCommand 55 * *
rbKeyWinLeftWindow * 91 * On “Natural” keyboards
rbKeyWinRightWindow * 92 * On “Natural” keyboards
rbKeyWinApplication 110 93 * On “Natural” keyboards
rbKeyQ 12 81 16
rbKeyW 13 87 17
rbKeyE 14 69 18
rbKeyR 15 82 19
rbKeyT 17 84 20
rbKeyY 16 89 21
rbKeyU 32 85 22
rbKeyI 34 73 23
rbKeyO 31 79 24
rbKeyP 35 80 25
rbKeyA * 65 30
rbKeyS 1 83 31
rbKeyD 2 68 32
rbKeyF 3 70 33
rbKeyG 5 71 34
rbKeyH 4 72 35
rbKeyJ 38 74 36
rbKeyK 40 75 37
rbKeyL 37 76 38
rbKeyZ 6 90 44
rbKeyX 7 88 45
rbKeyC 8 67 46
rbKeyV 9 86 47
rbKeyB 11 66 48
rbKeyN 45 78 49
rbKeyM 46 77 50
rbKey0 29 48 11
rbKey1 18 49 2
rbKey2 19 50 3
rbKey3 20 51 4
rbKey4 21 52 5
rbKey5 23 53 6
rbKey6 22 54 7
rbKey7 26 55 8
rbKey8 28 56 9
rbKey9 25 57 10
rbKeyPeriod 47 190 52
rbKeyComma 43 188 51
rbKeySlash 44 191 53 The key with /? generally next to right shift key.
rbKeyNum0 82 96 82 On numeric keypad or with NumLock
rbKeyNum1 83 97 79 On numeric keypad or with NumLock
rbKeyNum2 84 98 80 On numeric keypad or with NumLock
rbKeyNum3 85 99 81 On numeric keypad or with NumLock
rbKeyNum4 86 100 75 On numeric keypad or with NumLock
rbKeyNum5 87 101 76 On numeric keypad or with NumLock
rbKeyNum6 88 102 77 On numeric keypad or with NumLock
rbKeyNum7 89 103 71 On numeric keypad or with NumLock
rbKeyNum8 91 104 72 On numeric keypad or with NumLock
rbKeyNum9 92 105 73 On numeric keypad or with NumLock
rbKeyMultiply 67 106 55 On numeric keypad or with NumLock
rbKeyAdd 69 107 78 On numeric keypad or with NumLock
rbKeySubtract 78 109 74 On numeric keypad or with NumLock
rbKeyDivide 75 111 98 On numeric keypad or with NumLock
rbKeyDecimal 65 110 83 On numeric keypad or with NumLock
rbKeyNumEqual 81 * 117 On numeric keypad or with NumLock

from:http://classicteck.com/rbarticles/mackeyboard.php