构建win10下用IDEA运行Hadoop程序
本地解压hadoop安装包
如图

2.将hadoop添加到环境变量中


在Path变量中添加

下载winutils工具,将bin目录覆盖到解压的hadoop安装包中
3.在IDEA中构建新项目:
注意:IDEA必须下载专业版
打开之后:

如图

创建完项目之后,打开porn.xml文件:
添加配置项
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
| <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion>
<groupId>java_Hadoop</groupId> <artifactId>java_Hadoop</artifactId> <version>1.0-SNAPSHOT</version> <dependencies> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>2.8.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-hdfs</artifactId> <version>2.8.0</version> </dependency> </dependencies>
|
相关的包自己会从官网下载
具体实现代码:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
| package hdfs;
import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileStatus; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path;
import java.io.IOException;
public class ListHdfsFile {
private static Configuration configuration; private static FileSystem fs ;
static { configuration = new Configuration(); configuration.set("fs.defaultFS", "172.18.74.68:9000"); try { fs = FileSystem.get(configuration); } catch (IOException e) { e.printStackTrace(); }
}
private static void listFileDir(Path p) throws IOException { FileStatus[] statuses = fs.listStatus(p); for (FileStatus status : statuses) { Path path = status.getPath(); String name = path.getName(); System.out.println(name); } }
public static void main(String[] args) throws IOException { Path path = new Path("/"); listFileDir(path); } }
|