advertisement

Status Of Hpss

50 %
50 %
advertisement
Information about Status Of Hpss

Published on October 19, 2008

Author: guest3bd2a12

Source: slideshare.net

advertisement

Status of HPSS New Features, Requirements, and Installations Otis Graf IBM Global Services - Federal Houston, Texas October 1999

Topics HPSS New Features since Release 3.2 Next incremental updates Feature to be added in 4.2 Status of HPSS Installations Focus on Support Services Next Requirements Development Cycle

HPSS New Features since Release 3.2

Next incremental updates

Feature to be added in 4.2

Status of HPSS Installations

Focus on Support Services

Next Requirements Development Cycle

Summary of HPSS Releases and Patches

Includes through 4.1.1 (July 1999) File families Segregate files onto different groups of tapes. Storage Server uses family id passed in segment create calls for selecting the tape for the segment create Features Added Since Rel. 3.2 Level 1 F1 F2 FileA FileB (F1) FileC (F2) Level 2 Unassigned FileC FileB FileA

Includes through 4.1.1 (July 1999)

File families

Segregate files onto different groups of tapes.

Storage Server uses family id passed in segment create calls for selecting the tape for the segment create

Features Added Since Rel. 3.2 (Cont.) Filsets : A hierarchical collection of files and directories managed as a single unit. work play games play fs_goodwinj fs_teaff usr bin home goodwinj ... teaff Root Fileset junction junction

Filsets : A hierarchical collection of files and directories managed as a single unit.

Features Added Since Rel. 3.2 (Cont.) HPSS Filesets Managed by name server Can be either HPSS-only or HPSS/DFS DFS Filesets DFS-only HPSS/DFS archived HPSS & DFS name spaces not kept in synch Only way to get data is through DFS Performance similar to DFS-only if data is resident HPSS/DFS mirrored HPSS & DFS name spaces kept in synch Data can be seen and altered from either DFS or HPSS HPSS high speed data movement can be used

HPSS Filesets

Managed by name server

Can be either HPSS-only or HPSS/DFS

DFS Filesets

DFS-only

HPSS/DFS archived

HPSS & DFS name spaces not kept in synch

Only way to get data is through DFS

Performance similar to DFS-only if data is resident

HPSS/DFS mirrored

HPSS & DFS name spaces kept in synch

Data can be seen and altered from either DFS or HPSS

HPSS high speed data movement can be used

PFTP Enhancements Allows the use of multiple network and/or multiple systems for child processes for multi-network stripe transfers MPI-IO Interface Implements a subset of the MPI-2 standard Coordinates access to HPSS files from multiple processes Features Added Since Rel. 3.2 (Cont.)

PFTP Enhancements

Allows the use of multiple network and/or multiple systems for child processes for multi-network stripe transfers

MPI-IO Interface

Implements a subset of the MPI-2 standard

Coordinates access to HPSS files from multiple processes

Scaleable Accounting Accounting summary records support the following collection options during the accounting period: By account index & COS, Number of accesses Number of files transferred bytes used By account index, COS, & storage class number of files transferred number of accesses Features Added Since Rel. 3.2 (Cont.)

Scaleable Accounting

Accounting summary records support the following collection options during the accounting period:

By account index & COS,

Number of accesses

Number of files transferred

bytes used

By account index, COS, & storage class

number of files transferred

number of accesses

Performance Enhancements Support disk partitions > 2 GB Bigger virtual volumes (up to 16,384 segments per VV) Faster disk file creates and purges Support for Shelf Tapes Provide utility to identify and move volumes to shelf Mark cartridge metadata with shelf descriptor Generate operator mount requests via SSM pop-up Client API Enhancements Manually migrate/purge on a file basis Lock/unlock to prevent a file from being purged Features Added Since Rel. 3.2 (Cont.)

Performance Enhancements

Support disk partitions > 2 GB

Bigger virtual volumes (up to 16,384 segments per VV)

Faster disk file creates and purges

Support for Shelf Tapes

Provide utility to identify and move volumes to shelf

Mark cartridge metadata with shelf descriptor

Generate operator mount requests via SSM pop-up

Client API Enhancements

Manually migrate/purge on a file basis

Lock/unlock to prevent a file from being purged

Non-DCE client API (Solaris, IRIX & AIX) Non-DCE Mover Mover ported to other platforms Solaris (DCE) SGI IRIX (non-DCE) Additional Tape Drive Support Ampex DST 314 IBM 3590E Additional Library Support ADIC Automated Media Library (AML) Features Added Since Rel. 3.2 (Cont.)

Non-DCE client API (Solaris, IRIX & AIX)

Non-DCE Mover

Mover ported to other platforms

Solaris (DCE)

SGI IRIX (non-DCE)

Additional Tape Drive Support

Ampex DST 314

IBM 3590E

Additional Library Support

ADIC Automated Media Library (AML)

Number of HPSS Systems * *Notes: “ System” is a name space instance intended for production 1999 data is “year to date”.

*Notes:

“ System” is a name space instance intended for production

1999 data is “year to date”.

Listing of HPSS Sites as of October 1999

Listing of HPSS Sites (Cont.) as of October, 1999

Listing of HPSS Sites (Cont.) as of October, 1999

Current Activities Next scheduled patch (4.1.1.2 in Nov 99) Sun Phase 2 STK PVR and IBM 3494 PVR PFTP Client and Daemon MPI-IO SFS Backup utilities Some manageability improvements Follow-on patch (first quarter 00) IBM 3590 tape drives on Solaris StorageTek RAIT - testing and prototyping activity, no HPSS software change

Next scheduled patch (4.1.1.2 in Nov 99)

Sun Phase 2

STK PVR and IBM 3494 PVR

PFTP Client and Daemon

MPI-IO

SFS Backup utilities

Some manageability improvements

Follow-on patch (first quarter 00)

IBM 3590 tape drives on Solaris

StorageTek RAIT - testing and prototyping activity, no HPSS software change

Current Activities (Cont.) Development of Release 4.2 Planned date is 4th Quarter 00 Enhancement of On-Line Resources Tools repository On-line help (FAQ & problem/solution DB) On-line documentation (HTML & PDF) New Training Classes Advanced System Admin Class HPSS API programming class

Development of Release 4.2

Planned date is 4th Quarter 00

Enhancement of On-Line Resources

Tools repository

On-line help (FAQ & problem/solution DB)

On-line documentation (HTML & PDF)

New Training Classes

Advanced System Admin Class

HPSS API programming class

Current Activities (Cont.) Enhancement of Customer Support Better coordination with IBM Transarc Deployment/support procedures

Enhancement of Customer Support

Better coordination with IBM Transarc

Deployment/support procedures

Features in Rel. 4.2 Multiple Storage Subsystems * Comprised of Name Server, Bitfile Server, Migration Purge Server, & 1-n Storage Servers May be used to enhance concurrency May be used to partition servers to handle parts of the name space Data assigned to Storage Subsystem based on name Fileset creation established association with Name Server Junction attached fileset to name space *Note: HPSS Contract terms are pending for multiple name spaces.

Multiple Storage Subsystems *

Comprised of Name Server, Bitfile Server, Migration Purge Server, & 1-n Storage Servers

May be used to enhance concurrency

May be used to partition servers to handle parts of the name space Data assigned to Storage Subsystem based on name

Fileset creation established association with Name Server

Junction attached fileset to name space

*Note: HPSS Contract terms are pending for multiple name spaces.

Features in Rel. 4.2 (Cont.) Federated Name Space Defined as cooperative, distributed HPSS systems linked at the internal server level Supports use of all HPSS interfaces across multiple HPSS sites Maintains autonomy of individual HPSS installations Name spaces linked by junctions which point to filesets

Federated Name Space

Defined as cooperative, distributed HPSS systems linked at the internal server level

Supports use of all HPSS interfaces across multiple HPSS sites

Maintains autonomy of individual HPSS installations

Name spaces linked by junctions which point to filesets

Features in Rel. 4.2 (Cont.) Command Line Utilities Provide command line interface to SSM to enable management of HPSS from automated scripts Command-line program (hpssadm) which can be run interactively or in batch mode Allows user to manage servers, devices, storage classes, PVL jobs, and, to a limited extent, volumes

Command Line Utilities

Provide command line interface to SSM to enable management of HPSS from automated scripts

Command-line program (hpssadm) which can be run interactively or in batch mode

Allows user to manage servers, devices, storage classes, PVL jobs, and, to a limited extent, volumes

Features in Rel. 4.2 (Cont.) Non-DCE Client API Security Note: The majority of the non-DCE Client API has been developed for released in Release 4.1.1 Support client authorization / authentication by: None DCE (for platforms with DCE) Kerberos Provide Non-DCE Client Gateway (NDCG) with the ability to authenticate clients’ identities on a per connection basis

Non-DCE Client API Security

Note: The majority of the non-DCE Client API has been developed for released in Release 4.1.1

Support client authorization / authentication by:

None

DCE (for platforms with DCE)

Kerberos

Provide Non-DCE Client Gateway (NDCG) with the ability to authenticate clients’ identities on a per connection basis

Features in Rel. 4.2 (Cont.) Mass Configuration Provide mechanisms to create multiple server or drive / device records in a single request Provide mechanism to set persistent default values used in creating new configuration records A configurable default Log Policy will be provided, which may reduce the number of specific policies which need to be created

Mass Configuration

Provide mechanisms to create multiple server or drive / device records in a single request

Provide mechanism to set persistent default values used in creating new configuration records

A configurable default Log Policy will be provided, which may reduce the number of specific policies which need to be created

Features in Rel. 4.2 (Cont.) Gatekeeper Provides optional client interface to allow sites to schedule and monitor use of HPSS file resources Supports file create, open, stage, and close requests May be associated with one or many Storage Subsystems Sites will be able to implement site specific user level scheduling of storage requests. This may include: Limit the number of open files per user/host Prevent create requests for a user/host Collect statistics

Gatekeeper

Provides optional client interface to allow sites to schedule and monitor use of HPSS file resources

Supports file create, open, stage, and close requests

May be associated with one or many Storage Subsystems

Sites will be able to implement site specific user level scheduling of storage requests. This may include:

Limit the number of open files per user/host

Prevent create requests for a user/host

Collect statistics

Features in Rel. 4.2 (Cont.) Account Validation Maintain integrity of user account information. Users will only be able to use valid accounts Support cross-linked namespaces (remote sites). Individual sites select their own style of accounting While still supported, no longer be necessary to have user default account information in DCE registry Sites that do not need accounting or validation of user accounts may continue to run in the current fashion

Account Validation

Maintain integrity of user account information. Users will only be able to use valid accounts

Support cross-linked namespaces (remote sites). Individual sites select their own style of accounting

While still supported, no longer be necessary to have user default account information in DCE registry

Sites that do not need accounting or validation of user accounts may continue to run in the current fashion

Features in Rel. 4.2 (Cont.) Sun Solaris Phase 3 Port remaining servers Target OS is Solaris 8 HPSS back-end the IBM General Parallel File System (GPFS)

Sun Solaris Phase 3

Port remaining servers

Target OS is Solaris 8

HPSS back-end the IBM General Parallel File System (GPFS)

Post Release 4.2 Discussions No definitive release plans yet Features that were delayed from Rel. 4.2 Tape import/export Multiple distributed movers - allow multiple, possibly distributed movers to access a single device Begin new requirements development cycle: 2 Qtr 00 Features under discussion Improved manageability features Steps towards server consolidation and decreasing SFS dependencies Improved GPFS/HPSS performance through parallel data paths Backup/archive feature (file servers, workstations) Critical HEP requirements

No definitive release plans yet

Features that were delayed from Rel. 4.2

Tape import/export

Multiple distributed movers - allow multiple, possibly distributed movers to access a single device

Begin new requirements development cycle: 2 Qtr 00

Features under discussion

Improved manageability features

Steps towards server consolidation and decreasing SFS dependencies

Improved GPFS/HPSS performance through parallel data paths

Backup/archive feature (file servers, workstations)

Critical HEP requirements

Next Requirements Development Cycle Process Process begins at start of a new release cycle. Requirements contributed by customer and IBM. Requirements team consists of a representative from each customer site. Release contents are prioritized by requirements team. Developers responsible decomposing requirements into subsystem level.

Process

Process begins at start of a new release cycle.

Requirements contributed by customer and IBM.

Requirements team consists of a representative from each customer site.

Release contents are prioritized by requirements team.

Developers responsible decomposing requirements into subsystem level.

Requirements Development Cycle (Cont.) Process (Cont.) Requirements are costed, and then reprioritized as an iterative process. Customer balloting influences release contents, but is not binding. Project metrics used to determine the number of requirements accepted in the next baseline. Requirements not making baseline are included in the requirements document, but annotated as future release. Formal inspection of requirements is performed.

Process (Cont.)

Requirements are costed, and then reprioritized as an iterative process. Customer balloting influences release contents, but is not binding.

Project metrics used to determine the number of requirements accepted in the next baseline.

Requirements not making baseline are included in the requirements document, but annotated as future release.

Formal inspection of requirements is performed.

Requirements Development Cycle (Cont.) Process (Cont.) Any changes to the approved baseline require Technical and Executive Committee approval. Subsystem Requirements documents are published once a baseline is approved. Opportunity for the HEP Community HEP community develop common set of requirements Provide unified voice on requirements committee

Process (Cont.)

Any changes to the approved baseline require Technical and Executive Committee approval.

Subsystem Requirements documents are published once a baseline is approved.

Opportunity for the HEP Community

HEP community develop common set of requirements

Provide unified voice on requirements committee

Additional Resources HPSS User Forum Presentations www5.clearlake.ibm.com  “What’s New” Site Reports Details for current capabilities Details of Release 4.2 Details of DFSS and HPSS Multi-Platform Capability of HPSS Components www5.clearlake.ibm.com  “Product Information”  ”Multi-Platform Capability of HPSS” Core Servers and Movers User Interface Clients

HPSS User Forum Presentations

www5.clearlake.ibm.com  “What’s New”

Site Reports

Details for current capabilities

Details of Release 4.2

Details of DFSS and HPSS

Multi-Platform Capability of HPSS Components

www5.clearlake.ibm.com  “Product Information”  ”Multi-Platform Capability of HPSS”

Core Servers and Movers

User Interface Clients

Add a comment

Related pages

HPSS | Computational Information Systems Laboratory

HPSS. The High Performance Storage System (HPSS) ... Consult our Resource Status page for information about how to receive status updates automatically.
Read more

HPSS (High Performance Storage System) | Status.IU

We used your IP address to guess your location. If we got it wrong, please change your campus here for local results.
Read more

Managing Your Allocation and Your Users

Managing Your Users. For information on managing your users see the NIM Guide for ... After 5 months of Limited Status on HPSS the user is disabled on HPSS.
Read more

HPSS Status and Direction - California Institute of Technology

Slide 1 of 33 . Slide 1 of 33
Read more

HPSS Management Guide - hpss-collaboration.org

HPSS Configuration Overview ... HPSS Health and Status ..... 48 SM Server Connection Status Indicator ...
Read more

About - NERSC

The High Performance Storage System (HPSS) is a modern, flexible, performance-oriented mass storage system. It has been used at NERSC for archival storage ...
Read more

PPT - Status of HPSS PowerPoint Presentation - ID:5560472

Status of HPSS. New Features, Requirements, and Installations. Otis Graf IBM Global Services - Federal Houston, Texas October 1999. Topics. HPSS New ...
Read more

Laptops, Desktopsysteme, Drucker und mehr | HP® Deutschland

Informieren Sie sich über HP Drucker, Laptops, Desktopsysteme und mehr – auf der offiziellen HP® Website
Read more

Kunden - Wofür brauchen Sie einen HP Preferred Partner ...

2008 führte HP den Status "Gold" für HP Preferred Partner ein. Mit den HP Gold Preferred Partnern profitiert Ihre IT-Umgebung von höchster ...
Read more