利用ycsb测试hbase性能

浏览： ℃

字体：大中小

发布时间：2013-12-20 16:18:29

来源：

java 、maven、ycsb 的安装及配置见这篇博客：
http://blog.csdn.net/hs794502825/article/details/17309845

本篇博客主要介绍 hbase 的安装，以及利用 ycsb 对 hbase 进行基本的测试

步骤一：
在http://mirrors.hust.edu.cn/apache/hbase/下载hbase稳定版：
hbase-0.94.14.tar.gz 存放于 /home/hs/program 目录下
cd /home/hs/program
tar -zxvf hbase-0.94.14.tar.gz

步骤二：
修改/home/hs/program/YCSB/hbase目录下的pom.xml文件（根据你所使用的hbase版本和hadoop版本，注意红色部分）：
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.yahoo.ycsb</groupId>
<artifactId>root</artifactId>
<version>0.1.4</version>
</parent>
<artifactId>hbase-binding</artifactId>
<name>HBase DB Binding</name>
<dependencies>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>

<version>0.94.14</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>

<version>1.0.4</version>
</dependency>
<dependency>
<groupId>com.yahoo.ycsb</groupId>
<artifactId>core</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>${maven.assembly.version}</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>

步骤三：
因为更改了步骤二中的文件，所以需要重新编译YCSB项目
cd /home/hs/program/YCSB
mvn clean package | tee mvn-clean-package.dat //将输出同时输出到屏幕和文件
显示与下类似信息，则说明编译成功：
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] YCSB Root ............................................. SUCCESS [7.540s]
[INFO] Core YCSB ............................................. SUCCESS [13.293s]
[INFO] Cassandra DB Binding .................................. SUCCESS [15.861s]
[INFO] HBase DB Binding ...................................... SUCCESS [2:21.346s]
[INFO] Hypertable DB Binding ................................. SUCCESS [5.296s]
[INFO] DynamoDB DB Binding ................................... SUCCESS [7.347s]
[INFO] ElasticSearch Binding ................................. SUCCESS [18.140s]
[INFO] JDBC DB Binding ....................................... SUCCESS [5.495s]
[INFO] Mapkeeper DB Binding .................................. SUCCESS [4.054s]
[INFO] Mongo DB Binding ...................................... SUCCESS [3.368s]
[INFO] OrientDB Binding ...................................... SUCCESS [4.199s]
[INFO] Redis DB Binding ...................................... SUCCESS [3.261s]
[INFO] Voldemort DB Binding .................................. SUCCESS [4.113s]
[INFO] YCSB Release Distribution Builder ..................... SUCCESS [13.764s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4 minutes 8 seconds
[INFO] Finished at: Mon Dec 16 12:43:12 CST 2013
[INFO] Final Memory: 94M/227M

步骤四：
成功编译YCSB之后，在/home/hs/program/YCSB/distribution/target目录下会有ycsb-0.1.4.tar.gz
将该文件解压到/home/hs/program目录下：
tar -zxvf /home/hs/program/YCSB/distribution/target -C /home/hs/program

步骤五：
将/home/hs/program/hbase-0.94.14/lib中的以下文件：
slf4j-api-*.jar
slf4j-log4j12-*.jar
zookeeper-*.jar
拷贝到/home/hs/program/ycsb-0.1.4/hbase-binding/lib中

步骤六：
使用/home/hs/program/hbase-0.94.14/conf中的hbase-site.xml
替换掉/home/hs/program/ycsb-0.1.4/hbase-binding/conf中的hbase-site.xml

步骤七：
执行测试
终端1：
进入hbase的安装目录：
执行：./bin/start-hbase.sh
显示：
starting master, logging to /home/hs/program/hbase-0.94.14/bin/../logs/hbase-hs-master-hs-virtual-machine.out

接着执行：
./bin/hbase shell
create 'usertable','cf'

终端2中：
cd /home/hs/program/ycsb-0.1.4

hs@hs-virtual-machine:~/program/ycsb-0.1.4$ ./bin/ycsb load hbase -P workloads/workloada -p hosts=localhost -p columnfamily=cf > ./my-results/load-hbase-a

hs@hs-virtual-machine:~/program/ycsb-0.1.4$ ./bin/ycsb run hbase -P workloads/workloada -p hosts=localhost -p columnfamily=cf > ./my-results/run-hbase-a

得到如下结果：
YCSB Client 0.1
Command line: -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p hosts=localhost -p columnfamily=cf -t
[OVERALL], RunTime(ms), 5886.0
[OVERALL], Throughput(ops/sec), 169.89466530750934
[UPDATE], Operations, 487
[UPDATE], AverageLatency(us), 9366.180698151951
[UPDATE], MinLatency(us), 33
[UPDATE], MaxLatency(us), 4459758
[UPDATE], 95thPercentileLatency(ms), 0
[UPDATE], 99thPercentileLatency(ms), 0
[UPDATE], Return=0, 486
......
[READ], Operations, 514
[READ], AverageLatency(us), 2482.817120622568
[READ], MinLatency(us), 707
[READ], MaxLatency(us), 176125
[READ], 95thPercentileLatency(ms), 2
[READ], 99thPercentileLatency(ms), 29
[READ], Return=0, 514

>更多相关文章