资源简介
最近在研究hadoop与mapReduce,网上教程只有个wordcount程序示范,太简单,故写了个相对复杂点的涉及到多个文件之间同时运算的矩阵乘法的代码用于实验与测试,上传供大家学习与参考。
调用方法:
执行:hadoop jar matrix.jar com.baosight.mapred.test.MatrixMulti left:/tmp/a right:/tmp/b output:/tmp/c leftprefix:000000_ rightprefix:000000_
>参数1:left:左矩阵路径
>参数2:right:右矩阵路径
>参数3:output:左矩阵乘右矩阵结果路径
>参数4:leftprefix:左矩阵文件前缀
如/tmp/a/000000_0,000000_1,000000_2……的000000_
>参数5:rightprefix:右矩阵文件前缀
>左右矩阵按列存储,每个文件只存放一列值,如2x2的单位矩阵, 000000_0存放1 0
000000_1存放
0 1

代码片段和文件信息
/**
* Copyright 2012 宝信软件 解决方案事业部
*
* @author 樊后礼 f_houli@163.com fanhouli@baosight.com
*/
package com.baosight.mapred.test;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.WritableComparable;
public class MapInputKey implements WritableComparable{
public int leftColNum = 0;
public int rightColNum = 0;
public MapInputKey(){}
@Override
public void write(DataOutput out) throws IOException {
out.writeInt(leftColNum);
out.writeInt(rightColNum);
}
@Override
public void readFields(DataInput in) throws IOException {
leftColNum = in.readInt();
rightColNum = in.readInt();
}
@Override
public int compareTo(object o) {
MapInputKey other = (MapInputKey) o;
if (this.leftColNum == other.leftColNum
&& this.rightColNum == other.rightColNum) {
return 0;
} else {
return 1;
}
}
public boolean equals(object o) {
if (!(o instanceof MapInputKey))
return false;
MapInputKey other = (MapInputKey)o;
return (this.leftColNum == other.leftColNum && this.rightColNum == other.rightColNum);
}
}
属性 大小 日期 时间 名称
----------- --------- ---------- ----- ----
文件 1197 2012-06-27 01:19 src\com\baosight\mapred\test\MapInputKey.java
文件 787 2012-06-27 01:20 src\com\baosight\mapred\test\MapInputValue.java
文件 2738 2012-06-27 01:20 src\com\baosight\mapred\test\MatrixInputFormat.java
文件 2769 2012-06-27 01:20 src\com\baosight\mapred\test\MatrixInputRecordReader.java
文件 1641 2012-06-27 01:20 src\com\baosight\mapred\test\MatrixInputSplit.java
文件 1347 2012-06-27 01:20 src\com\baosight\mapred\test\MatrixMapper.java
文件 2702 2012-06-27 01:20 src\com\baosight\mapred\test\MatrixMulti.java
文件 947 2012-06-27 01:20 src\com\baosight\mapred\test\MatrixReducer.java
文件 2932 2012-06-27 01:20 src\com\baosight\mapred\test\ReduceInputKey.java
文件 867 2012-06-27 01:20 src\com\baosight\mapred\test\ReduceOutputValue.java
目录 0 2012-06-27 01:29 src\com\baosight\mapred\test\
目录 0 2012-06-27 01:09 src\com\baosight\mapred\
目录 0 2012-06-27 01:09 src\com\baosight\
目录 0 2012-06-27 01:09 src\com\
目录 0 2012-06-12 07:52 src\
- 上一篇:android下圆角列表的实现带事件监听
- 下一篇:Android精美登录界面设计
相关资源
- 大数据hadoop,spark,hive等等面试汇总
- 《Hadoop with Python》 pdf
- 绝对可用hadoop-eclipse-plugin-1.1.2.jar
- hadoop-common-2.7.1-bin
- hadoop2.5.0 snappy编译jar包
- Hadoop 2.7.4 Windows 64位 编译bin包含win
- hadoop-hdfs-2.7.7.jar
- 修复版 hadoop-0.21.0-eclipse-plugin.jar
- hadoop-eclipse-plugin-1.2.1.jar
- Native Hadoop3.2.1 Library 64位编译
- hadoop-common-2.7.3.jar
- avro-1.8.1
- Hadoop2.7.7配合使用的winutil
- hadoop 32位lib包
- hadoop电商网站分析系统毕业设计项目
- hadoop-eclipse-plugin-2.7.5.jar275778
- Hadoop-Eclipse插件2.7.6
- JavaWeb操作hadoop2.6 HDFS从页面上传,,
- hadoop-eclipse-plugin-2.7.3.jar 插件
- hadoop-eclipse-plugin-2.7.2完美兼容版
- hadoop-eclipse-plugin-2.6.4.jar
- MapReduce分析年气象数据用源代码
- hadoop-eclipse-plugin-2.8.5.jar
- hadoop-eclipse-plugin-3.1.2.jar
- Hadoop的jar包
- hadoop-eclipse-plugin-2.9.2.jar 插件
- hadoop-eclipse-plugin-2.7.7.jar 插件
- hadoop-3.1.3.tar.gz
- hadoop-eclipse-plugin-2.7.3.jar
- hadoop-eclipse-plugin-2.6.0.jar
评论
共有 条评论