class Google::Apis::DataprocV1::HadoopJob

A Cloud Dataproc job for running Hadoop MapReduce jobs on YARN.

Attributes

archive_uris[RW]
Optional

HCFS URIs of archives to be extracted in the working directory of

Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or . zip. Corresponds to the JSON property `archiveUris` @return [Array<String>]

args[RW]
Optional

The arguments to pass to the driver. Do not include arguments, such

as `-libjars` or `-Dfoo=bar`, that can be set as job properties, since a collision may occur that causes an incorrect job submission. Corresponds to the JSON property `args` @return [Array<String>]

file_uris[RW]
Optional

HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to

the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks. Corresponds to the JSON property `fileUris` @return [Array<String>]

jar_file_uris[RW]
Optional

Jar file URIs to add to the CLASSPATHs of the Hadoop driver and

tasks. Corresponds to the JSON property `jarFileUris` @return [Array<String>]

logging_config[RW]

The runtime logging config of the job. Corresponds to the JSON property `loggingConfig` @return [Google::Apis::DataprocV1::LoggingConfig]

main_class[RW]

The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in `jar_file_uris`. Corresponds to the JSON property `mainClass` @return [String]

main_jar_file_uri[RW]

The HCFS URI of the jar file containing the main class. Examples: 'gs://foo- bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test- samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop- mapreduce-examples.jar' Corresponds to the JSON property `mainJarFileUri` @return [String]

properties[RW]
Optional

A mapping of property names to values, used to configure Hadoop.

Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code. Corresponds to the JSON property `properties` @return [Hash<String,String>]

Public Class Methods

new(**args) click to toggle source
# File generated/google/apis/dataproc_v1/classes.rb, line 852
def initialize(**args)
   update!(**args)
end

Public Instance Methods

update!(**args) click to toggle source

Update properties of this object

# File generated/google/apis/dataproc_v1/classes.rb, line 857
def update!(**args)
  @main_jar_file_uri = args[:main_jar_file_uri] if args.key?(:main_jar_file_uri)
  @main_class = args[:main_class] if args.key?(:main_class)
  @args = args[:args] if args.key?(:args)
  @jar_file_uris = args[:jar_file_uris] if args.key?(:jar_file_uris)
  @file_uris = args[:file_uris] if args.key?(:file_uris)
  @archive_uris = args[:archive_uris] if args.key?(:archive_uris)
  @properties = args[:properties] if args.key?(:properties)
  @logging_config = args[:logging_config] if args.key?(:logging_config)
end