Tuesday 15 February 2011

Hadoop mapreduce python command line arguments -


In my Python Mapper code, I need to reach the 'path' in the 'path'. How can it be used in Python code?

You can read the input file from os.environ . For example,

  import os input_file = os.environment ['map_input_file']   

In fact, you read other JobConf from os You can. Environment . Note: During the execution of a streaming job, the names of the "mapread" parameter are changed. Dots (.) Underscore (_) For example, mapred.job.id becomes mapred_job_id and mapred.jar becomes mapred_jar. Use the underscores to parameter names to get values ​​in the streaming job mapper / reducer. See.

I also get a very useful post for you:

No comments:

Post a Comment