资源简介
这里是coursera课程Hadoop Platform and Application Framework的所有项目源代码,都通过测试考核,所以应该准确无误。原课程讲解Hadoop和Spark,有兴趣的小伙伴们可以参考,不过建议自己编程,有助于能力的提高。原课程的讲义以及软件开发环境安装等就不再附上,可以去官网免费下载,我这里只附上官网没有的代码。

代码片段和文件信息
#!/usr/bin/env python
#the above just indicates to use python to intepret this file
# ---------------------------------------------------------------
#This mapper code will input a line of text and output
#
# ---------------------------------------------------------------
import sys #a python module with system functions for this OS
# ------------------------------------------------------------
# this ‘for loop‘ will set ‘line‘ to an input line from system
# standard input file
# ------------------------------------------------------------
for line in sys.stdin:
#-----------------------------------
#sys.stdin call ‘sys‘ to read a line from standard input
# note that ‘line‘ is a string object ie variable and it has methods that you can apply to it
# as in the next line
# ---------------------------------
line = line.strip() #strip is a method ie function associated
# with string variable it will strip
# the carriage return (by default)
keys = line.split() #split line at blanks (by default)
# and return a list of keys
for key in keys: #a for loop through the list of keys
value = 1
print(‘{0}\t{1}‘.format(key value) ) #the {} is replaced by 0th1st items in format list
#also note that the Hadoop default is ‘tab‘ separates key from the value
属性 大小 日期 时间 名称
----------- --------- ---------- ----- ----
目录 0 2015-12-30 14:30 code\
目录 0 2015-12-30 03:15 code\041 assignment wordcount\
文件 8080 2015-12-29 21:40 code\041 assignment wordcount\041 readme.txt
文件 3749 2015-12-29 22:12 code\041 assignment wordcount\A note on debugging mapreduce programs.txt
文件 41 2015-12-29 21:22 code\041 assignment wordcount\testfile1
文件 29 2015-12-29 21:22 code\041 assignment wordcount\testfile2
文件 1460 2015-12-29 21:17 code\041 assignment wordcount\wordcount_mapper.py
文件 100 2015-12-29 21:38 code\041 assignment wordcount\wordcount_num0_output.txt
文件 1963 2015-12-29 21:20 code\041 assignment wordcount\wordcount_reducer.py
目录 0 2015-12-30 03:15 code\042 assignment join1\
文件 1363 2015-12-30 02:16 code\042 assignment join1\042 readme.txt
文件 37 2015-12-29 23:08 code\042 assignment join1\join1_FileA.txt
文件 122 2015-12-29 23:08 code\042 assignment join1\join1_FileB.txt
文件 1580 2015-12-29 22:36 code\042 assignment join1\join1_mapper.py
文件 3953 2015-12-29 22:39 code\042 assignment join1\join1_reducer.py
目录 0 2015-12-30 03:16 code\043 assignment join2\
文件 1368 2015-12-30 02:16 code\043 assignment join2\043 readme.txt
文件 1714 2015-12-29 23:22 code\043 assignment join2\join2_genchanA.txt
文件 3430 2015-12-29 23:22 code\043 assignment join2\join2_genchanB.txt
文件 5152 2015-12-29 23:22 code\043 assignment join2\join2_genchanC.txt
文件 17114 2015-12-29 23:22 code\043 assignment join2\join2_gennumA.txt
文件 34245 2015-12-29 23:22 code\043 assignment join2\join2_gennumB.txt
文件 51400 2015-12-29 23:22 code\043 assignment join2\join2_gennumC.txt
文件 1216 2015-12-30 01:11 code\043 assignment join2\join2_mapper.py
文件 370 2015-12-30 03:02 code\043 assignment join2\join2_output.txt
文件 1017 2015-12-30 02:59 code\043 assignment join2\join2_reducer.py
文件 333 2015-12-29 23:20 code\043 assignment join2\make_data_join2.txt
文件 1188 2015-12-29 23:22 code\043 assignment join2\make_join2data.py
文件 14739 2015-12-30 01:39 code\043 assignment join2\original command aide.txt
目录 0 2015-12-30 04:31 code\051 assignment spark\
文件 513 2015-12-30 03:50 code\051 assignment spark\051 Install IPython.txt
............此处省略5个文件信息
- 上一篇:基于labview的软键盘
- 下一篇:经典的基本PSO粒子群优化算法的测试函数
相关资源
- 《Hadoop大数据技术》课程设计报告.
- Hadoop.in.Practice.2nd.Edition
- 基于Greenplum Hadoop- 分布式平台的大数
- 构建用户自画像视频教程真实企业项
- hdp安装手册
- hadoop技术内幕三件套
- [百度网盘]Hadoop技术内幕 深入解析M
- Hadoop技术内幕 深入解析YARN架构设计与
- Hadoop技术内幕:深入解析YARN架构设计
- Hadoop技术内幕 完整三本合集baidu链接
- 中国大数据技术与产业发展报告
- 在Hadoop生态中大数据平台架构与实践
- hadoop集群搭建教程.zip
- Hadoop技术内幕 深入解析HADOOP COMMON和
- 大数据技术之面试题 .pdf
- windows64位平台的hadoop2.8.4插件包(ha
- hadoop-2.5.0-cdh5.3.6 编译native包
- MapReduce基于物品的协同过滤算法实现
- 大数据组件介绍PPT
- 《Hadoop构建数据仓库实践》电子书
- Gi盘——基于hadoop的分布式网盘项目
- Hadoop深度学习
- CDH5hadoop集群搭建手册,大数据平台搭
- 第8讲:Pig应用开发文档
- 第7讲:Hive数据仓库文档
- 基于Hadoop的微博信息挖掘
- hadoop API
- Hadoop编程操作 高性能云计算
- Hadoop安全:大数据平台隐私保护.pdf
- Kerberos权威指南 Kerberos The Definitive G
评论
共有 条评论