解决Windows.createDirectoryWithMode0

解决Windows.createDirectoryWithMode0Hadoop安装解决Windows.createDirectoryWithMode0

解决Windows.createDirectoryWithMode0

最近在学习Hadoop ，在window本地运行Hadoop时，老是出现 java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode0(Ljava/lang/String;I)V ，通过自己的各种尝试，终于解决了这个问题。

Hadoop安装

开发环境：Win10； Hadoop版本：hadoop-2.6.0-cdh5.16.2.tar 安装步骤

以管理员身份启动WinRAR 解压 hadoop-2.6.0-cdh5.16.2.tar 解压到指定目录，我的安装目录是： D:\WorkTools\Hadoo;务必以管理员身份运行，否则在解压的过程中会出现错误。配置环境变量：HADOOP_HOME=D:\WorkTools\Hadoop，path=%HADOOP_HOME%/bin；配置%HADOOP_HOME%\etc\hadoop\core-site.xml；<configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:8020</value> </property> </configuration> 配置%HADOOP_HOME%\etc\hadoop\hdfs-site.xml；<configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.permissions</name> <value>false</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>/E:/WorkSpaces/Repositories/Hadoop/Data/NameNode</value> </property> <property> <name>fs.checkpoint.dir</name> <value>/E:/WorkSpaces/Repositories/Hadoop/Data/snn</value> </property> <property> <name>fs.checkpoint.edits.dir</name> <value>/E:/WorkSpaces/Repositories/Hadoop/Data/snn</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>/E:/WorkSpaces/Repositories/Hadoop/Data/DataNode</value> </property> </configuration> 配置%HADOOP_HOME%\etc\hadoop\mapred-site.xml；<configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration> 配置%HADOOP_HOME%\etc\hadoop\yarn-site.xml;<configuration> <!-- Site specific YARN configuration properties --> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> </configuration> 配置%HADOOP_HOME%\etc\hadoop\hadoop-env.cmd;set JAVA_HOME=D:/WorkTools/Oracle/Java/jdk1.8.0_201 初始化 ； 在CMD中执行：hdfs namenode -format 启动 在CMD中执行：start-dfs.cmd start-yarn.cmd 验证 HDFS : localhost:50070 YARN : localhost:8088

解决Windows.createDirectoryWithMode0

添加 winutils 下载页面：https://github.com/steveloughran/winutils建议采用 git clone git@github.com:steveloughran/winutils.git 的方式； 第一次我选择的是2.6.0的，但是没有解决问题，第二次选择3.0.0的。复制hadoop-3.0.0/bin中的全部文件去替换%HADOOP_HOME%/bin中的文在要运行的类中添加静态代码块。static { try { System.load("D:/WorkTools/Hadoop/bin/hadoop.dll");//建议采用绝对地址。 } catch (UnsatisfiedLinkError e) { System.err.println("Native code library failed to load.\n" + e); System.exit(1); } } 类似于这样：public class Demo { static { try { System.load("D:/WorkTools/Hadoop/bin/hadoop.dll"); } catch (UnsatisfiedLinkError e) { System.err.println("Native code library failed to load.\n" + e); System.exit(1); } } public static void main(String[] args) throws Exception{ 启动程序验证。Counters: 30 File System Counters FILE: Number of bytes read=858955041 FILE: Number of bytes written=27104946 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 Map-Reduce Framework Map input records=300000 Map output records=300000 Map output bytes=3600000 Map output materialized bytes=4200036 Input split bytes=954 Combine input records=0 Combine output records=0 Reduce input groups=1 Reduce shuffle bytes=4200036 Reduce input records=300000 Reduce output records=1 Spilled Records=600000 Shuffled Maps =6 Failed Shuffles=0 Merged Map outputs=6 GC time elapsed (ms)=6 Total committed heap usage (bytes)=3909615616 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=173576072 File Output Format Counters Bytes Written=19