Hadoop-Request

From Helpme

Jump to: navigation, search

Contents

Overview

The resources available on the Engineering Hadoop cluster are limited. All Hadoop cluster usage proposals must be submitted to the ECSC for consideration. Proposals must come from Engineering faculty. Only approved proposals will be granted access.

Proposal outline

All proposals need to answer the following questions, at a minimum:

  • What is the overall usage purpose? (Teaching, Research, Senior Design, etc)
  • How many users will need cluster access?
  • How much usable HDFS space will be required? (Either in total or per-user)
  • What Hadoop technologies will be required? (ie, HDFS, MapReduce, Spark, Mahout, etc)
  • What is the desired start date for the proposed usage? (Usage proposals must be submitted at least three months in advance of their desired start date)
  • When can the accounts and data associated with this proposal be deleted?
  • Please provide a detailed description of your intended use of the cluster.

Proposal Recommendations

Proposals are evaluated on numerous criteria, but a few key considerations are outlined below.

Disk Space

Disk space is the single most limited resource in the cluster. The less disk space a proposal requires, the easier it will be to approve.

Duration

Proposals with short, clearly defined durations will be easier to approve than those of long or indefinite durations.

Resource Limits

A brief overview of the cluster configuration can be found here. Proposals requiring only a small percentage of available cluster resources will be easier to approve.

Proposal Submission

Proposals should be submitted to: hadoop-request@engr.scu.edu

Requestors will be notified of the ECSC's decision at the e-mail address from which they submit their proposal.

Personal tools