Monitoring and Debugging a Hadoop Job on Amazon Elastic MapReduce
You can connect to Hadoop Job Tracker on Elastic MapReduce Cluster and look at it, without having to get your ssh key for the cluster, log in to the node and then run the hadoop binaries to query the job tracker with textual output.
Karmasphere Studio's Elastic MapReduce Cluster Monitor and Job Monitor functionality provides information about all the temporary and permanent clusters that have been invoked and the cluster capacity and its usage by a job.
The Elastic MapReduce cluster monitor is invoked automatically whenever you run a job. Or you can open it by right-clicking on your cluster and selecting "Monitor".
Note that the Elastic MapReduce Cluster Monitor shows information about all jobs on Amazon Elastic MapReduce associated with the account credentials you used to configure the cluster object.
This is useful for multiple engineers working on the same project and to examine the status and information about jobs that are running, failed or completed.
Note that you can also monitor and debug jobs that you may have run through the Elastic MapReduce console or other tools.
The top pane of the general cluster window shows attributes of each job you have deployed or run from the Elastic MapReduce console.
The bottom pane "Summary", "Steps" and "Logs" functionality and the "Terminate...", "File System Browser...""Hadoop Monitor..." buttons are specific to the job you currently have selected in the top pane.
Summary
This enables you to see summary information about the job selected in the top pane.
Logs
This enables you to browse and view the contents of log files for your job.
Terminate
This enables you to terminate the running job you have selected in the top pane.
Hadoop Monitor
If you press the "Hadoop Monitor..." button, you can discover more information about the cluster capacity and job usage information about Task Trackers, Accounting, Map and Reduce Attempts.
To see more detail about an individual job, double-click a job in the "Jobs" section of the display. This brings up the Job Monitor window and shows you additional job details.
File System Browser
This enables you to browse the HDFS running on the Elastic MapReduce cluster, via ssh.