UsingTheHCPClientTools PDF

Hitachi Content Platform
Using the HCP Client Tools
FASTFIND LINKS Document Organization Product Version Getting Help Table of Contents
MK-96ARC005-10
Copyright 20072011 Hitachi Data Systems Corporation, ALL RIGHTS RESERVED No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for any purpose without the express written permission of Hitachi Data Systems Corporation (hereinafter referred to as Hitachi Data Systems). Hitachi Data Systems reserves the right to make changes to this document at any time without notice and assumes no responsibility for its use. This document contains the most current information available at the time of publication. When new and/or revised information becomes available, this entire document will be updated and distributed to all registered users. Some of the features described in this document may not be currently available. Refer to the most recent product announcement or contact your local Hitachi Data Systems sales office for information about feature and product availability. Notice: Hitachi Data Systems products and services can be ordered only under the terms and conditions of the applicable Hitachi Data Systems agreement(s). The use of Hitachi Data Systems products is governed by the terms of your agreement(s) with Hitachi Data Systems. By using this software, you agree that you are responsible for: a) Acquiring the relevant consents as may be required under local privacy laws or otherwise from employees and other individuals to access relevant data; and b) Ensuring that data continues to be held, retrieved, deleted, or otherwise processed in accordance with relevant laws. Hitachi is a registered trademark of Hitachi, Ltd. in the United States and other countries. Hitachi Data Systems is a registered trademark and service mark of Hitachi, Ltd. in the United States and other countries. All other trademarks, service marks, and company names are properties of their respective owners.
Contents
Preface........................................................................................................vii
Intended audience . . . . Product version . . . . . . Document organization . Syntax notation . . . . . . Related documents. . . . Getting help. . . . . . . . . Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... ... ... ... ... ... ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... .... .... .... .... .... .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii . vii . viii . .ix ..x . xii . xii
Introduction to Hitachi Content Platform.............................................1-1

About Hitachi Content Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2 Namespaces and tenants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2 Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3
Introduction to the HCP client tools ....................................................2-1

HCP client tools overview . . . . . . . . . . . . HCP client tools command conventions. . . Keyword parameters and arguments . Keyword prefixes . . . . . . . . . . . . . Command-line structure . . . . . . . . Case sensitivity. . . . . . . . . . . . . . . Common parameters . . . . . . . . . . . . File specifications and directory paths. About the examples in this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .... .... .... .... .... .... .... .... .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2 2-3 2-3 2-4 2-4 2-4 2-5 2-5 2-6
Working with the HCP client tools ......................................................3-1

HCP client tools and alternatives HCP Data Migrator . . . . . . . Operating system tool sets . Selecting the tool to use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2 3-2 3-3 3-3 iii
Contents Using the HCP Client Tools
Data movement workflow . . . . . . . . . . . . . . Task 1: Identifying the data . . . . . . . . . Task 2: Deciding where to put the data . Task 3: Copying or moving the data . . . Using scripts . . . . . . . . . . . . . . . . . . . . Workflow examples . . . . . . . . . . . . . . . . . .
. . . . . .
. . . . . .
.... .... .... .... .... ....
.... .... .... .... .... ....
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
3-4 3-4 3-5 3-6 3-6 3-7
Finding files (arcfind) ..........................................................................4-1

About arcfind . . . . . . . . . . . . . . . . . . . . How arcfind works . . . . . . . . . . . . . arcfind output . . . . . . . . . . . . . . . . arcfind log . . . . . . . . . . . . . . . . . . . arcfind syntax . . . . . . . . . . . . . . . . . . . arcfind parameter descriptions. . . . . . . . arcfind command considerations . . . . . . Specifying dates and times . . . . . . . Datetime format . . . . . . . . . . . . . Using multiple time specifications . Text strings . . . . . . . . . . . . . . . . . . arcfind output as arcmv input . . . . . arcfind parameter interactions . . . . . arcfind examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... . . . . . . . . . . . . . . .... .... .... .... .... .... .... .... .... .... .... .... .... .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2 . 4-2 . 4-3 . 4-3 . 4-5 . 4-5 . 4-9 . 4-9 .4-10 .4-10 .4-11 .4-11 .4-12 .4-13
Moving files (arcmv) ...........................................................................5-1

About arcmv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . arcmv input . . . . . . . . . . . . . . . . . . . . . . . . . . . arcmv output . . . . . . . . . . . . . . . . . . . . . . . . . . arcmv log . . . . . . . . . . . . . . . . . . . . . . . . . . . . arcmv with SSL security . . . . . . . . . . . . . . . . . . arcmv with HCP namespaces . . . . . . . . . . . . . . . arcmv syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . arcmv parameter descriptions . . . . . . . . . . . . . . . . . arcmv command considerations . . . . . . . . . . . . . . . . Source and target file locations . . . . . . . . . . . . . Constructing source and target paths . . . . . . . Using single and multiple sources and targets . Example of location construction . . . . . . . . . . Using HCP namespaces . . . . . . . . . . . . . . . . . . . Using arcfind with arcmv . . . . . . . . . . . . . . . . . . Namespace access protocol considerations . . . . . URL encoding. . . . . . . . . . . . . . . . . . . . . . . . . . ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... ..... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2 . 5-3 . 5-4 . 5-5 . 5-7 . 5-8 . 5-9 .5-10 .5-17 .5-17 .5-18 .5-19 .5-20 .5-20 .5-21 .5-22 .5-22
iv
Metadata properties . . . . . . . . . . . . . . . . . . . . . . HCP namespace metadata properties . . . . . . . . Default namespace metadata properties . . . . . . Namespace directory structure. . . . . . . . . . . . . . . New directories . . . . . . . . . . . . . . . . . . . . . . . . . Optimizing performance . . . . . . . . . . . . . . . . . . . Retrying on failure . . . . . . . . . . . . . . . . . . . . . . . arcmv errors . . . . . . . . . . . . . . . . . . . . . . . . . . . arcmv parameter interactions . . . . . . . . . . . . . . . Overwriting using the same source and destination Custom metadata considerations . . . . . . . . . . . . . arcmv examples . . . . . . . . . . . . . . . . . . . . . . . . . . . .
....... ....... ....... ....... ....... ....... ....... ....... ....... directory ....... .......
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
.5-22 .5-23 .5-23 .5-24 .5-24 .5-25 .5-26 .5-26 .5-26 .5-27 .5-27 .5-28
Creating directories (arcmkdir) ...........................................................6-1

About arcmkdir . . . . . . . . . . . . . . . . . arcmkdir syntax . . . . . . . . . . . . . . . . . arcmkdir parameter descriptions . . . . . arcmkdir command considerations . . . . Reserved directory name. . . . . . . . arcmkdir target location processing arcmkdir examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ........... ........... ........... ........... ........... ........... ........... .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2 6-2 6-3 6-4 6-4 6-4 6-5
HCP client tools use cases.................................................................7-1

Automatic image storage . . The challenge . . . . . . . The solution. . . . . . . . . The Python script . . . . . Time-critical image storage . The challenge . . . . . . . The solution. . . . . . . . . The shell script. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . .... .... .... .... .... .... .... .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-2 7-2 7-3 7-3 7-4 7-5 7-5 7-6
Appendix: Installing the HCP client tools ....................................Appendix-1

Compiled-executable platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix-2 Installing the executables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix-2 Compiling the client tools source code . . . . . . . . . . . . . . . . . . . . . . . . Appendix-3
Glossary Index
vi
Preface
This book contains all the information you need to use the set of command-line client tools distributed with Hitachi Content Platform (HCP). For information on installing the client tools, see Managing a Tenant and its Namespaces or Managing the Default Tenant and Namespace. Note: Throughout this book, the word Unix is used to represent all UNIXlike operating systems (such as UNIX itself or Linux), except where Linux is specifically required.
Intended audience
This book is intended for IT professionals who are responsible for storing corporate data in an HCP namespace. It assumes you:
Are familiar with the file systems and namespace access protocols in
use at your site
Have experience using command-line interfaces Have a basic understanding of HCP technology
Product version
This document applies to release 4.1 of HCP.
Preface Using the HCP Client Tools
vii
Document organization
Document organization
This book contains seven chapters and a glossary.
Chapter/Appendix
Chapter 1, Introduction to Hitachi Content Platform Chapter 2, Introduction to the HCP client tools Chapter 3, Working with the HCP client tools
Description
Introduces basic HCP concepts that you need to understand in order to successfully use the HCP client tools Introduces the HCP client tools and explains conventions and examples used in the rest of the book Explains how the HCP client tools fit into the data storage workflow and discusses other tools that provide the same functionality Presents an overview, syntax, command considerations, and examples of the arcfind tool Presents an overview, syntax, command considerations, and examples of the arcmv tool Presents an overview, syntax, command considerations, and examples of the arcmkdir tool Describes two real-life use cases in which customers are successfully using the HCP client tools
Chapter 4, Finding files (arcfind)
Chapter 5, Moving files (arcmv)
Chapter 6, Creating directories (arcmkdir) Chapter 7, HCP client tools use cases
Appendix, Installing the HCP client tools Describes how to install compiled HCP client tool files and to client tool source code
viii
Syntax notation
Syntax notation
The table below describes the conventions for HCP client tool command syntax.
Notation boldface Meaning
Type exactly as it appears in the syntax (if the context is case insensitive, you can vary the case of the letters you type) Replace with a value of the indicated type Vertical bar Choose one of the elements on either side of the bar, but not both Square brackets Include none, one, or more of the elements between the brackets Parentheses Include exactly one of the elements between the parentheses Replace with the combination of the directory path and name of a file Replace with a directory path with no file name Ellipsis Optionally, repeat the preceding parameter as many times as needed
Example
This book shows: arcfind You enter: arcfind
italics |
This book shows: directory-path You enter: HR/Benefits/Health This book shows: arcfind -h|--help You enter: arcfind -h or: arcfind --help This book shows:
[ ]
arcfind [--dirs] input-directory-path

You enter: arcfind --dirs HR/Benefits or: arcfind HR/Benefits This book shows: You enter: arcfind -e BenDirFind.log or: arcfind --logfile BenDirFind.log
( )
arcfind [(-e|--logfile) log-file-spec]
-file-spec
This book shows: output-file-spec You enter: CopyResults\Financials-2007-02-28-out.txt
-path ...
This book shows: input-directory-path You enter: HR/Benefits This book shows:
arcfind input-directory-path ...

You enter: arcfind HR/Benefits HR/Contacts
ix
Related documents
Related documents
The following documents contain additional information about Hitachi Content Platform:
Administering HCP This book explains how to use an HCP system to

monitor and manage a digital object repository. It discusses the capabilities of the system, as well as its hardware and software components. The book presents both the concepts and instructions you need to configure the system, including creating the tenants that administer access to the repository. It also covers the processes that maintain the integrity and security of the repository contents.
Managing a Tenant and Its Namespaces This book contains complete

information for managing the HCP tenants and namespaces created in an HCP system. It provides instructions for setting up both administrative user accounts and data access accounts, configuring the HTTP protocol, which allows access to namespaces, managing search, and downloading installation files for HCP Data Migrator and the HCP client tools. It also explains how to work with retention classes and the privileged delete functionality.
Managing the Default Tenant and Namespace This book contains

complete information for managing the default tenant and namespace in an HCP system. It provides instructions for changing tenant and namespace settings, configuring the protocols that allow access to the namespace, managing search, and downloading installation files for HCP Data Migrator and the HCP client tools. It also explains how to work with retention classes and the privileged delete functionality.
Replicating Tenants and Namespaces This book covers all aspects of

tenant and namespace replication. Replication is the process of copying tenants and namespaces from one HCP system to another to ensure data availability and enable disaster recovery. The book describes how replication works, contains instructions for working with replication links, and explains how to monitor the replication process.
HCP Management API Reference This book contains the information

you need to use the HCP management API. This REST API enables you to create and manage tenants and namespaces programmatically. The book explains how to use the API to access an HCP system, specify resources, and update and retrieve resource properties.
Related documents
Using a Namespace This book describes the properties of objects in

HCP namespaces. It provides instructions for accessing namespaces by using the HTTP protocol for the purpose of storing, retrieving, and deleting objects, as well as changing object metadata such as retention and shred settings. It also explains how to manage namespace content and view namespace information in a web browser.
Using the Default Namespace This book describes the file system
HCP uses to present the contents of the default namespace. It provides instructions for accessing the namespace by using the HCP-supported protocols for the purpose of storing, retrieving, and deleting objects, as well as changing object metadata such as retention and permissions.
Searching Namespaces This book describes the HCP Seach Console.

It explains how to search namespaces for objects that satisfy criteria you specify. It also explains how to manage and manipulate queries and search results. The book contains many examples, which you can use as models for your own searches.
Using HCP Data Migrator This book contains the information you
need to install and use the HCP Data Migrator (HCP-DM) utility distributed with HCP. This utility enables you to copy data between local file systems, HCP namespaces, and earlier HCAP archives. It also supports bulk delete operations. The book describes both the interactive window-based interface and the set of command-line tools included in HCP-DM.
Installing an HCP System This book provides the information you

need to install the software for a new HCP system. It explains what you need to know to successfully configure the system and contains step-by-step instructions for the installation procedure.
Third-Party Licenses and Copyrights This book contains copyright

and license information for third-party software distributed with or embedded in HCP.
Installing an HCP 500 System Final On-site Setup This book

contains instructions for deploying an assembled and configured HCP 500 system at a customer site. It explains how to make the necessary physical connections and reconfigure the system for the customer computing environment.
Installing an HCP 300 System Final On-site Setup This book

contains instructions for deploying an assembled and configured HCP 300 system at a customer site. It explains how to make the necessary physical connections and reconfigure the system for the customer computing environment.
xi
Getting help
Getting help
The Hitachi Data Systems customer support staff is available 24 hours a day, seven days a week. If you need technical support, please call:
United States: (800) 446-0744 Outside the United States: (858) 547-4526
Note: If you purchased HCP from a third party, please contact your authorized service provider.
Comments
Please send us your comments on this document: hcp.documentation.feedback@hds.com Include the document title, number, and revision, and refer to specific sections and paragraphs whenever possible. Thank you! (All comments become the property of Hitachi Data Systems.)
xii
1
Introduction to Hitachi Content Platform
Hitachi Content Platform (HCP) is a distributed storage system designed to support large, growing repositories of fixed-content data. An HCP repository is partitioned into namespaces, each of which stores both data and metadata about that data. This chapter provides a brief overview of some of the concepts you need to understand in order to successfully use the HCP client tools.
Introduction to Hitachi Content Platform Using the HCP Client Tools
11
About Hitachi Content Platform
About Hitachi Content Platform

Hitachi Content Platform is the distributed, fixed-content, data storage system from Hitachi Data Systems. It provides a cost-effective, scalable, easy-to-use repository that can accommodate all types of data, from simple text files to medical images to multigigabyte database images. A fixed-content storage system is one in which the data cannot be modified. HCP uses write-once, read-many (WORM) storage technology, internal processes, and various kinds of metadata to ensure the integrity of the stored data. It also provides easy access to the repository for adding, retrieving, and, when allowed, deleting the stored data.
Namespaces and tenants

An HCP repository is partitioned into namespaces. A namespace is a logical grouping of files such that the files in one namespace are not visible in any other namespace. Namespaces provide a mechanism for separating the data stored for different applications. For example, one namespace could be used for accounts receivable while another is used for accounts payable. HCP and default namespaces An HCP system can have multiple HCP namespaces and one default namespace. To access an HCP namespace, users and applications must present valid credentials. Access to the default namespace is open to all users and applications. New applications are typically written against HCP namespaces. The default namespace is typically used for legacy applications. The root of the directory structure for files in each HCP namespace is a directory named rest. The root directory in the default namespace is fcfs_ data. Tenants Namespaces are owned and managed by administrative entities called tenants. A tenant typically corresponds to an organization, such as a company or a division or department within a company. A tenant can also correspond to an individual person. Each tenant, except one, can own multiple namespaces. The exception, called the default tenant, owns the default namespace and only that namespace.
12
Metadata
Metadata
Data has metadata, both before you store it in a namespace and once its in the namespace. Metadata is information about data. Outside HCP, files have POSIX metadata such as creation and access times. In HCP, files have system metadata and custom metadata. System metadata consists of HCP-specific information such as retention and shred settings and, for default namespaces only, POSIX metadata. Custom metadata consists of user-defined properties. By default, when you add a file to an HCP namespace, it inherits system metadata settings from the namespace configuration. When you add a file to the default namespace, it inherits system metadata settings from the directory you store it in. For more information on metadata, see Using a Namespace or Using the Default Namespace.
13
Metadata
14
2
Introduction to the HCP client tools
The HCP client tools are a set of command-line tools provided with Hitachi Content Platform. These tools allow you to store and manage files in namespaces. This chapter describes how the client tools fit into HCP. It also explains the command conventions and examples used in this book.
Introduction to the HCP client tools Using the HCP Client Tools
21
HCP client tools overview
HCP client tools overview

The HCP client tools are a component of HCP. They provide a means to easily populate a namespace with large amounts of data. The tools The HCP client tools let you copy or move data:
The arcfind tool generates a list of the files you want to copy or
move. Files can be in a local or remote system or a default namespace. arcfind cannot find files in HCP namespaces.
The arcmv tool copies or moves the files in a list from a location on a
client computer or from a namespace to another location on a client computer or namespace. arcmv is a high-performance, multithreaded tool designed to handle large numbers of files at a time.
The arcmkdir tool creates one or more empty directories in a local or

remote file system or in a default namespace. arcmkdir cannot create directories in HCP namespaces. For information on what differentiates the HCP client tools from tools with similar capabilities, see HCP client tools and alternatives on page 3-2. Namespace access protocols With the HCP client tools, the client computer communicates with HCP by using a namespace access protocol. The tools support these protocols:
HTTP and WebDAV offer a simple, fast programmatic interface to

HCP and are suitable for transferring large amounts of data.
CIFS and NFS offer full file-system semantics with no programming

required. Because of their rich feature sets, these protocols are not appropriate for transferring large amounts of data. Note: HCP namespaces support only the HTTP protocol. Support includes HTTP with SSL (HTTPS).
22
HCP client tools command conventions
The big picture The figure below shows the relationship between the HCP client tools and HCP.
Local file system
HCP
Remote file system
HCP client tools arcfind arcmv arcmkdir
Supported protocols HTTP/HTTPS WebDAV CIFS NFS

Each HCP client tool command line begins with the name of the tool you want to execute. The rest of the line consists of parameters that tell the tool what you want to do. The following sections describe general rules that apply to using the tools. For the conventions this book uses to show the syntax of each command, see the Preface. For the syntax and parameter descriptions for the individual commands, see the chapter for the applicable tool.
Keyword parameters and arguments

The parameters for the HCP client tool commands are either keyword parameters or arguments:
A keyword parameter consists of a text string you enter as is (the

keyword) and, often, a value you supply. In the example below, the keyword is --logfile and the user-supplied value is ..\PDFfiles-hr-201010-09.log:
--logfile ..\PDFfiles-hr-2010-10-09.log
Some keyword parameters can occur multiple times in a single command line. When including a recurring keyword parameter in a command line, you need to include the keyword in each occurrence.
23
An argument is a value you supply that tells the tool what to work on,
and is not preceded by a keyword. In the HCP client tool commands, only arcfind and arcmkdir take arguments, and in both cases, the arguments are one or more directory paths separated by spaces; for example:
HR/Benefits HR/Contacts
The arcmv command does not take explicit arguments. Instead, it works on a list of files contained in the file specified by the --infile keyword parameter, or input from stdin.
Keyword prefixes
Each keyword in an HCP client tool command line is preceded by a single hyphen (-) if it consists of a single letter or two hyphens (--) if it consists of two or more letters. You can concatenate single-letter keyword parameters with each other as long as none before the last one take user-supplied values. When you do this, you drop the hyphens between the letters. For example, in an arcfind command, you can concatenate the -v, -L, and -e parameters like this:
-vLe ..\PDFfiles-hr-2010-10-09.log
Tip: When concatenating single-letter options, be sure to precede the string with a single hyphen. If you use a double hyphen, the tool treats the entire string as one keyword.
Command-line structure
In an HCP client tool command line, keyword parameters always come before any arguments. The keyword parameters themselves, however, can be in any order. Most keyword parameters are optional, and you can include none, one, or more of them.
arcmv and arcmkdir each have one required keyword parameter. These
tools dont execute if you omit that parameter from the command line. You use spaces to separate the parameters in HCP client tool commands.
Case sensitivity
The keywords in HCP client tool command lines are case sensitive in both Windows and Unix. The command names, however, and file and directory specifications are case sensitive in Unix, as expected, but not in Windows.
24
Common parameters
Some command parameters are available in more than one tool. All three tools, for example, support the --help and --version parameters:
--help causes the tool to display its syntax rules. --version causes the tool to display its version number.
If you issue an HCP client tool command without any parameters, the tool displays both its help and version number. Likewise, if the command line contains a syntax error or the --help parameter anywhere in it, the tool displays both its help and version number and doesnt execute, even if other parameters are valid. Additional common keyword parameters deal with logging. They are:
--quiet (in arcfind and arcmv) --verbose (in all three tools) --logfile (in arcfind and arcmv)
File specifications and directory paths

In the syntax in this book, each item ending in -file-spec is a file name with or without its directory path. Items ending in -path are directory paths with no final delimiter (for example, HR\Benefits, not HR\Benefits\). In either case, unless otherwise indicated in the parameter descriptions, the directory paths in the HCP client tool command lines can be either absolute or relative. HCP file specifications are limited to 255 bytes in each segment and 4,095 maximum for the entire string, including separators. All characters are valid within each segment, except the forward slash (ASCII 47), which is the separator character, and NULL (ASCII 0 (zero)). Additionally:
To specify a plus sign (+) in a URL, use %2b or %2B. To specify a percent sign (%) in a URL, use %25.
25
About the examples in this book
When a file specification or directory path refers to a local or remote file system, use the conventions specific to the platform youre working from. For example, in Windows, use the backslash (\) as the separator in directory paths; in Unix, use the forward slash (/). When the path identifies an HTTP or WebDAV location, use only forward slashes. Note: HTTP automatically converts backslashes in Windows file specifications and directory paths to forward slashes before sending them to HCP.
About the examples in this book

The examples in this book are for the most part presented in both Windows and Unix. In each case, the examples use the conventions of their respective platforms. For very short examples, however, the book generally shows only one version, Windows.
26
3
Working with the HCP client tools
You can use the HCP client tools as well as tools that are native to your operating system or access protocol to copy or move data from a client to HCP. Whichever tools you choose, however, the workflow remains the same. You need to find the data, decide where it goes, and put it there. This chapter discusses the kinds of tools you need and where they fit into the workflow of copying or moving data to an HCP namespace.
Working with the HCP client tools Using the HCP Client Tools
31
HCP client tools and alternatives

HCP client tools perform find, copy, move, and create directory functions, but they arent the only tools that do this. They are simply one alternative that may or may not be the best choice in any given circumstance.
HCP Data Migrator

HCP provides HCP Data Migrator (HCP-DM), a utility that can copy and delete files and directories. HCP-DM has several advantages over the client tools, but also lacks some of their features, such the ability to find files based on selection criteria. HCP-DM has these features, which are not available in the client tools:
It has both GUI and command-line interfaces. It supports bulk deletes in the GUI and provides an hcpdm delete
command that can do batch deletes.
It uses namespace profiles, which conveniently encompass all

information needed to access a specific namespace, including the URL and login credentials for HCP namespaces. You can use the GUI or a command line tool to manage these profiles.
The GUI interface supports drag-and-drop copying within the GUI and
From Windows Explorer to the GUI.
The GUI lets you monitor the progress of copy and delete operations.
The displayed information includes metrics such as files transferred per second, succeeded and failed copies, and elapsed time. This information can be particularly helpful in tracking large migration jobs.
The GUI lets you interrupt, save, and resume jobs. You can resume
jobs that have been intentionally interrupted or that could not complete due to errors.
The GUI lets you lets you dynamically configure the load on the HCP
system and lets you configure a reduced load for specific times of day. For information on some of the limitations that HCP-DM has compared to the individual client tool commands, see Selecting the tool to use on page 3-3.
32
Operating system tool sets

Windows and Unix have their own tool sets that overlap the HCP client tools in what they can do. Many applications and utilities also come with some of the same capabilities. Additionally, you can write scripts using specific access protocols to perform the operations you want.
Selecting the tool to use

So why use HCP client tools? arcfind
arcfind offers multiplatform support from a single command syntax. HCPDM does not provide a find facility, but you can use arcfind output as input to the hcpdm move command. Other tools you can use to generate lists of files, such as the platform-specific Unix ls and Windows dir commands and
application-specific database queries, have more limited capabilities. arcmv arcmv, which provides both copy and move capabilities, is multithreaded, resulting in a high rate of throughput. arcmv supports multiple data transfer protocols with the same command syntax. With arcmv, you can map source locations to target locations with different names and paths, thereby allowing the HCP namespace to have a different directory structure from that of any of the data sources. Additionally, when overwriting a file, arcmv first deletes the existing file, then writes the new one, in accordance with WORM semantics. Because HCP has permission and retention settings that affect file deletion, files you cannot delete are protected from being overwritten. HCP-DM provides an hcpdm copy command that is similar to arcmv but provides a different and more limited set of parameters and features. Here are some of the more significant limitations on hcpdm copy:
It does not include an overwrite capability. It cannot delete files from the source after they have been copied, but
you can use the hcpdm delete command to do so.
It cannot require the source files to have custom metadata files and can
apply only a single custom metadata file to all files copied from the local file system.
It does not have as flexible or complete logging facilities as arcmv.
33
Data movement workflow
It cannot preserve the source atime and mtime values for files copied to
the default namespace. The values for copies are always the time they are added to the namespace. To decide which command is more appropriate for your purposes, you should compare the command descriptions in detail. Other tools you can use to copy or move a group of files, such as the single-operation copy and move commands and curl, which has a protocoldependent syntax, have more limited capabilities. arcmkdir
arcmkdir is a multiplatform, multiprotocol tool for creating empty directories. You can use it to create directories on local and remote file systems and in a default namespace. HCP-DM does not provide an equivalent command, but you can use the HCP-DM GUI to create empty directories. The Windows and Unix mkdir commands are limited to local and remote file systems.

To copy or move data into an HCP namespace, you need to perform three tasks: 1. Identify the data to be copied or moved 2. Decide where to put the data in the namespace 3. Copy or move the data to the namespace
Task 1: Identifying the data

Before you can copy or move data, you need to decide which data you want. Then you need to generate a list of the files containing that data. This list can be written to a text file, piped directly into a copy or move command, or entered interactively for a copy or move operation. Finding the files with the data you want may be as simple as selecting the directory theyre in. However, more often than not, you need to select files that meet some specific set of criteria based on properties such as:
Creation date or last modification date Location in the file system hierarchy
34
Partial file name or file name extension File size Person who created the file Organization that owns the file Time period covered by the file (for example, first quarter or yesterday)
Which tool you use for this task may depend partially on the types of search criteria the tool supports. For other factors that can affect your choice of tool, see arcfind on page 3-3. For considerations regarding lists of files generated by arcfind for use with the arcmv tool, see arcfind output as arcmv input on page 4-11.
Task 2: Deciding where to put the data

A well-organized namespace requires planning. The directory structure in the namespace typically is not a mirror image of the structure the data is copied or moved from. Some tools, including arcmv, automatically create new directories when the paths of the files being copied or moved include directories that dont already exist in the namespace. When using tools that do this, you need to ensure that any directories created by the operation dont deviate from the conventions established for the namespace directory structure. Some tools expect the directories named in their command lines to already exist, even if those directories are not populated. For tools like these, you need to create the empty directories ahead of time. Once you create these directories, you can link to them from the client computer. Which tool you use to create new directories may depend partially on where you want to put them. For more information on this, see arcmkdir on page 3-4. Tip: When deciding where to put files in a default namespace, be sure to consider the ownership, permissions, and HCP-specific metadata for the target directory. For information on these properties, see Using the Default Namespace.
35
Task 3: Copying or moving the data

When you have the list of files to copy or move, you can proceed with that operation. Which tool you choose for this task may depend partially on how you want to use it. For example, some tools are faster than others; some tools are better than others for use in scripts; some tools let you enter the list of files interactively; and some tools provide better reporting of what theyve done. For other factors that can affect your choice of tool, see arcmv on page 3-3. For the default namespace, you also need to decide which protocol to use for the copy or move operation. You can consider factors like these to help you choose:
Which protocols are available and how much traffic theyre currently
experiencing.
How closely management of the data is tied to an application (for

example, you cannot use the HCP client tools with files stored in a document management system built in a database). If you cannot manipulate the data outside the application, you may need to use application-specific tools to copy or move it. These tools may each work with a limited set of protocols.
How much data youre copying or moving. The faster the protocol, the
better it can handle large amounts of data without affecting other HCP activities.
Using scripts
Scripts enable you to put multiple commands together so you dont need to enter them individually. Putting copy and move operations into a script has these advantages:
You can use a Windows scheduled task or a Unix cron job to run the
script automatically at particular times (for example, at 1:00 a.m. every night). Tip: When large amounts of data are being copied or moved, run the script when the demand for network resources is low.
You can use variables such as SYSTEMROOT and DATE to make the
script more reusable.
36
Workflow examples
You can set the base directory so relative paths in the script work every
time it runs.
You can ensure that the copy and move commands are well-formed
each time theyre invoked.
Workflow examples
The sample scripts below the first for Windows, the second for Unix show the workflow for backing up files from a client computer to an HCP namespace. The scripts use arcfind to locate the files to be copied and pipe the resulting list of files directly into arcmv. You can schedule scripts like these to create daily backups. Windows example Heres the sample script:
set DATE=2010-10-24 arcfind --ctime "+%DATE% 00:05:00" --minsize 1 -e findlog.txt ^ -v \Benefits | arcmv --src "C:" --dst http://finance.europe.hcp.^ example.com/rest --dstroot "Backup-%DATE%" -v -l 800 ^ --logfile Backup-log.%DATE%.txt
Heres what this script does: 1. Sets the DATE variable to 2010-10-24 2. Invokes arcfind, which:
a. Searches in C:\Benefits for files with metadata that was last modified
after 12:05 a.m. on 2010-10-24 and that contain at least one byte of data
b. Writes session and file statistics to a file named findlog.txt in the
current working directory

c. Pipes the list of files it generates into arcmv
Note: The vertical bar (|) is the symbol for piping. The caret (^) is the line continuation character. 3. Invokes arcmv, which:
a. Looks on the C: drive of the client computer for the files being piped
in.
37
Workflow examples b. Writes the files it finds to HCP starting at this base location: http://finance.europe.hcp.example.com/rest/Backup-2010-10-24
The remainder of the file paths consist of the source file paths, starting with \Benefits.
c. Writes file statistics to stderr and file statistics, session statistics, all
informational, warning, and error messages, with some additional processing information to a file named Backup-log.2010-10-24.txt in the current working directory Unix example Heres the sample script:
#!/bin/sh DATE='date +%F' FINDLOG=findlog arcfind --ctime "+$DATE 00:05:00" --minsize 1 -e $FINDLOG /benefits \ | arcmv --dst http://finance.europe.hcp.example.com/rest \ --dstroot "Backup-$DATE" -v -l 800 --logfile Backup-log.$DATE
Heres what this script does: 1. Sets the DATE variable to the current date with the format yyyy-mmdd 2. Sets the FINDLOG variable to findlog 3. Invokes arcfind, which:
a. Searches /benefits for files with metadata that was last modified
after 12:05 a.m. on the current date and that contain at least one byte of data
b. Writes session and file statistics to a file named findlog in the
current working directory

c. Pipes the list of files it generates into arcmv
Note: The vertical bar (|) is the symbol for piping. The backslash (\) is the line continuation character.
38
Workflow examples
4. Invokes arcmv, which:

a. Uses the paths piped from the arcfind command return data as
the paths for source files.

b. Writes the files it finds to HCP starting at this base location: http://finance.europe.hcp.example.com/rest/Backup-value-of-DATE-variable
DATE is the value set in step 1. The remainder of the file paths consist of the source file paths, starting with /benefits.
c. Writes file statistics to stderr and file statistics, session statistics,
all informational, warning, and error messages, with some additional processing information to a file named Backup-log.valueof-the-DATE-variable in the current working directory, where DATE is the value set in step 1
39
Workflow examples
310
4
Finding files (arcfind)
The arcfind tool lets you generate lists of files or directories that you can use as input to other tools, such as arcmv. In the arcfind command line, you specify criteria that arcfind uses to select the items youre interested in. arcfind searches for these items only within the directory trees you identify in the command line. This chapter presents instructions for using arcfind. Note: You cannot use arcfind to find files in HCP namespaces.
Finding files (arcfind) Using the HCP Client Tools
41
About arcfind
About arcfind
The arcfind tool searches for files or directories in one or more directory trees. In the command line to invoke the tool, you specify the top-level directory of each tree you want to search. You can identify the directory either with an absolute path or the path relative to the current working directory. You cannot use arcfind to search a namespace with HTTP or WebDAV. To use arcfind, you need to map or mount the namespace so it appears to be part of the native file system. By default, arcfind returns a list of files, but you can use a command-line parameter to get a list of directories instead. Tip: You can pipe the output from arcfind directly into the arcmv tool. As compared with creating and saving a file, piping makes more efficient use of the resources on your computer. It also makes the arcfind and arcmv commands simpler and saves you keystrokes. Important: If you plan to pipe the output from arcfind directly into the arcmv tool, be sure to read arcfind output below, arcfind output as arcmv input on page 4-11, and Source and target file locations on page 5-17 before you construct each command.
How arcfind works

arcfind walks down each directory tree you specify and selects the files or
directories that match the criteria you supply using command-line parameters. Using these parameters, you can filter files or directories based on time attributes. Additionally, you can filter files based on their size or by comparing their names with one or more text strings you specify. You can also use command-line parameters to restrict the search to only a portion of each directory tree. You can do this in two ways:
By limiting the search to a specified number of levels in the directory

trees
By eliminating branches from the search based on the occurrence of

one or more specified text strings in the directory names
42
About arcfind
arcfind output
Output from arcfind is a list of files or directories. You can specify whether the list is written to a file or displayed on-screen. Within the list, the path shown for each item starts with the path of the top-level directory exactly as it appears in the command line. For example, if you specify the top-level directory as test/test_1, the resulting list of files would look something like this:
test/test_1/file_a test/test_1/file_b test/test_1/test_2/file_c test/test_1/test_2/file_d test/test_1/test_X test/test_1/Z_test/file_k test/test_1/Z_test/test_7/temp_file
Tip: In Windows, use WordPad rather than Notepad to open arcfind output files. In WordPad, each listed file appears in a separate line. In Notepad, the files are listed in a single line.
arcfind log
In addition to the output list, arcfind generates a log that contains information about its processing. You can specify whether the log is written to a file or displayed on-screen, as well as what type of information it contains. Tip: In Windows, use WordPad rather than Notepad to open arcfind log files. In WordPad, each listed file appears in a separate line. In Notepad, the files are listed in a single line. Statistics in the arcfind log The statistics in the arcfind log are labeled for easy search and script postprocessing. The table below describes these labels.
Label
Directories processed pruned max depth
Description
The number of directories arcfind encountered as it walked down the directory trees The number of those directories arcfind did not process because of --prune parameters in the arcfind command line The number of levels arcfind walked down the directory trees
43
About arcfind
(Continued)
Label
max passing entries Files processed passed filtered others
Description
The highest number of files arcfind selected from a single directory The number of files arcfind encountered as it walked down the directory trees The number of encountered files that met the criteria specified in the arcfind command line The number of encountered files arcfind filtered out based on the criteria specified in the command line The number of encountered files arcfind did not process for any other reason (for example, because what appeared to be a data file was actually a socket or device file) The total size, in KiB, of the files that passed The average size, in KiB, of the files that passed See Distribution of file sizes below
Total passed size Avg/file Distribution of File Sizes
Distribution of file sizes As part of its statistics, an arcfind log includes the distribution of the selected files by size. The distribution ranges are:
< 1KiB 1KiB - 10KiB 10KiB - 100KiB 100KiB - MiB 1MiB - 10MiB 10MiB - 100MiB 100MiB - 1GiB 1GiB - 10GiB 10GiB - 100GiB >= 100GiB
In these ranges:
KiB = 1024 bytes MiB = 10242 bytes GiB = 10243 bytes
44
arcfind syntax
arcfind syntax
Heres the syntax for the arcfind command:
arcfind (-h|--help) |--version |([--dirs] [-L|--link] [--maxdepth directory-level-count] [(--ctime [+|-|=]posix-ctime)...] [(--atime [+|-|=]posix-atime)...] [(--mtime [+|-|=]posix-mtime)...] [--minsize minimum-byte-count] [--maxsize maximum-byte-count] [--size exact-byte-count] [(--exclude exclude-string)...] [(--prune prune-string)...] [--sjis] [(-o|--outfile) output-file-spec] [(-e|--logfile) log-file-spec] [-q|--quiet] [-v|--verbose] input-directory-path ...)
arcfind parameter descriptions

The table below describes the parameters for the arcfind command.
Parameter (--atime [+|-|=]posix-atime)... Description
Searches only for files with a POSIX atime value that is after (+), before (-), or equal to (-) the specified time. If you omit this parameter altogether, arcfind searches for files or directories regardless of their atime values. For information on specifying the parameter, including using multiple --atime parameters to specify ranges, see Specifying dates and times on page 4-9. Note: By default, HCP doesnt change the value of the atime attribute. However, the system can be configured to synchronize atime values with retention settings, and users and applications can change atime attribute values. For information on atime synchronization with retention settings, see Using the Default Namespace.
45

(Continued)
Parameter (--ctime [+|-|=]posix-ctime)...
Description
Searches only for files with a POSIX ctime value that is after (+), before (-), or equal to (-) the specified time. If you omit this parameter altogether, arcfind searches for files or directories regardless of their ctime values. For information on specifying the parameter, including using multiple --ctime parameters to specify ranges, see Specifying dates and times on page 4-9.
--dirs
Returns a list of directories instead of a list of files. If you omit this parameter, arcfind returns a list of files. Tip: The --dirs parameter is useful when arcfind is used in conjunction with arcmkdir.
(-e|--logfile) log-file-spec
Writes diagnostics and statistics to the specified file. log-file-spec identifies a text file. If the file doesnt exist, arcfind creates it. If the file already exists, arcfind overwrites it without notifying you. In either case, the directory in which the file is located must already exist. For information on specifying the log file, see File specifications and directory paths on page 2-5. If you omit this parameter, arcfind sends log information to stderr. When youre using arcfind interactively, stderr is the window youre working in.
(--exclude exclude-string)... Searches only for files with names that do not include the
specified case-sensitive text string. For information on specifying text strings, see Text strings on page 4-11. If you omit this parameter, arcfind doesnt exclude files based on their names. Note: This parameter does not apply to directory names.
-h|--help -L|--link
Displays and describes the syntax for the arcfind command; then exits. Tells arcfind to follow symbolic links when searching a directory tree. If you omit this parameter, arcfind doesnt follow directory paths past any symbolic links it encounters. Note: This parameter is meaningful only on Unix systems. On Windows systems, arcfind always follows symbolic links.
46

(Continued)
Parameter --maxdepth directory-level-count
Description
Searches only the specified number of levels down the directory trees, where the top directory in each is level one. For example, if the directory tree is A>B>C>D and --maxdepth is 3, arcfind returns a list of the files in A, B, and C or, if you include the --dirs parameter, arcfind returns a list containing the directories A, B, and C. Valid values for directory-level-count are integers greater than or equal to 1 (one). If you omit this parameter, arcfind searches each entire directory tree.
--maxsize maximum-byte-count
Searches only for files with a size less than or equal to the specified number of bytes. Valid values for maximum-bytecount are integers greater than or equal to 0 (zero). If you omit this parameter, arcfind searches for files without checking for a maximum size.
--minsize minimum-byte-count
Searches only for files with a size greater than or equal to the specified number of bytes. Valid values for minimum-bytecount are integers greater than or equal to 0 (zero). If you omit this parameter, arcfind searches for files without checking for a minimum size.
(--mtime [+|-|=]posix-mtime)...
Searches only for files with a POSIX mtime value that is after (+), before (-), or equal to (-) the specified time. If you omit this parameter altogether, arcfind searches for files or directories regardless of their mtime values. For information on specifying the parameter, including using multiple --mtime parameters to specify ranges, see Specifying dates and times on page 4-9. Note: HCP doesnt change the value of the mtime attribute. However, users and applications can change this value.
47

(Continued)
Parameter (-o|--outfile) output-file-spec
Description
Writes the list of selected files or directories to the specified file. output-file-spec identifies a text file. If the file doesnt exist, arcfind creates it. If the file already exists, arcfind overwrites it without notifying you. In either case, the directory in which the file is located must already exist. For information on specifying the output file, see File specifications and directory paths on page 2-5. If you omit this parameter, arcfind sends the list to stdout. When youre using arcfind interactively, stdout is the window youre working in. Tip: When you send the arcfind output to stdout, you can pipe it directly into the arcmv tool.
(--prune prune-string)...
Searches only directories with names that do not include the specified case-sensitive text string. For information on specifying text strings, see Text strings on page 4-11. When a directory is excluded, arcfind doesnt search its subdirectories either. If you omit this parameter, arcfind doesnt exclude directories based on their names.
-q|--quiet
Directs arcfind not to log processing and diagnostic information. This parameter is ignored if you specify the -v (--verbose) parameter. Searches only for files with a size exactly equal to the specified number of bytes. Valid values for exact-byte-count are integers greater than or equal to 0 (zero). If you omit this parameter, arcfind doesnt exclude files based on an exact size.
--size exact-byte-count
--sjis
Tells arcfind to check each input directory path for Shift JIS encoding. If arcfind encounters a path thats encoded that way, it converts the path to UTF-8 and continues processing. Logs detailed processing information in addition to the diagnostics and statistics generated by default. Displays the version number of the currently installed arcfind tool; then exits.
-v|--verbose --version
48
arcfind command considerations

(Continued)
Parameter input-directory-path ...
Description
Specifies the directory trees arcfind should search in a local or mounted file system. The path cannot be an HTTP or WebDAV URL. Use spaces to separate multiple directory paths. If a path includes spaces, enclose it in double quotation marks ("). The parameters used in an arcfind command apply to all the specified input directories. For information on specifying the input directory paths, see File specifications and directory paths on page 2-5 and arcfind output as arcmv input on page 4-11.

The following sections present considerations for using the arcfind command.
Specifying dates and times

The --atime, --ctime, and --mtime parameters take a parameter in this form:
[+|-|=]posix-datetime
A datetime value starting with + matches times after the specified value. A datetime value starting with - matches times before the specified value. A datetime value starting with = matches only the specified date and time. This behavior is the default.
For details on specifying the date and time, see Datetime format on page 4-10. You can combine parameters to set time ranges. For details, see Using multiple time specifications on page 4-10.
49
Datetime format
The datetime format for the --atime, --ctime, and --mtime parameters is YYYY-MM-DD[ hh:mm:ss], where:
YYYY is the four-digit year. MM is the two-digit month. DD is the two-digit day. hh is the hour on a twenty-four hour clock. mm is the number of minutes past the hour. ss is the number of seconds past the minutes.
The year, month, and day, are separated by hyphens (-); the hour, minutes, and seconds, by colons (:). If you omit the time, arcfind uses 00:00:00 (midnight). The date and time must be separated by a space. When including the time, you need to either put the entire time specification in double quotation marks (""), including the leading character, if any (for example, "+2010-10-09 07:30:00"), or use a platform-specific escape character in front of the space (for example, +2010-10-09\ 07:30:00).
Using multiple time specifications

Multiple time specifications are cumulative. That is, the file must meet all conditions specified in all parameters. As a result:
If you combine --mtime, --ctime, and --atime parameters, arcfind

returns the names of files with time values that meet all the specified criteria.
If you specify two parameters of the same type (for example, atime),
one with the plus sign (+) and one with the minus sign (-), arcfind returns the names of files with time values between the specified times.
If you specify two or more parameters of the same type (for example,
atime) with values starting with +, arcfind returns the names of files
with time values after the latest parameter value.
410
If you specify two or more parameters of the same type (for example,
atime) with values starting with -, arcfind returns the names of files with time values before the earliest parameter value.
Also, if you specify an --mtime or --atime value along with a --ctime value, the specified --ctime value must be earlier than the --mtime or --atime value. If the --ctime value is later than either of these, arcfind returns an empty list.
Text strings
The text string in an --exclude or --prune parameter can be any combination of alphanumeric and special characters allowed by the platform-specific rules for files and directory names. If the string includes one or more spaces, you need to either put the entire string in double quotation marks (for example, "test C") or use a platform-specific escape character in front of each space (for example, test\ C). Text strings in Unix are case sensitive; those in Windows are not.
arcfind output as arcmv input

arcfind starts each line in its output with the input directory path you
specified in the command line. If this is an absolute path, each path in the output is absolute as well. When you use the arcfind output as input to an arcmv command, arcmv generates the source and destination paths for the move operation by appending each path in the arcfind output to the source and destination locations specified in the arcmv command. The following conditions apply to the combination of paths:
If the output of the arcfind command consists of absolute paths and the
source location specified in the arcmv command is an absolute path, the resulting paths are invalid.
If you use two periods (..) to represent parent directories in the input
directory path for arcfind, each line in the output file begins with the double period. When arcmv concatenates these paths with a source location, the result is an invalid path. The paths specified in the two commands must combine correctly. For example:
411
The arcfind output must consist of paths that are relative to any source
directory specified by --src and --srcroot parameters in the arcmv command. Before running the arcfind command, you may need to set the working directory to ensure that the arcfind command generates output that the arcmv command can use.
When running arcfind in Windows, if you include the drive letter in the
input directory path, the drive letter is included in each file path in the output file. When you use this output as input to arcmv, the resulting file paths have the drive letter embedded in them (with no preceding delimiter). To prevent this problem, run arcfind on the drive on which the files you want are located so you dont need to include the drive letter in the input directory path. For more information about using the two commands together, see Using arcfind with arcmv on page 5-21. For examples that use arcfind output directly as input to an arcmv command, see Workflow examples on page 3-7.
arcfind parameter interactions

arcfind lists only files and directories that meet all of the criteria specified
by the command parameters. As a result, some combinations of parameters can produce unexpected results:
If you specify a --maxdepth of 1 along with the --dirs parameter, arcfind

lists only the directory at the top of the tree.
If you specify a --maxdepth with symbolic links enabled, and an item at

the specified --maxdepth level is a symbolic link to a directory, arcfind doesnt search that directory.
If you specify both the --minsize and --maxsize parameters, the

specified --minsize value must be less than the specified --maxsize value. If the --minsize value is greater than the --maxsize value, arcfind returns an empty list.
If you include the --size parameter with the --minsize and/or --maxsize
parameters, arcfind returns an error.
If you specify multiple --exclude parameters, arcfind searches only for

files with names that do not include any of the specified text strings.
If you specify multiple --prune parameters, arcfind searches only

directories with names that do not include any of the specified text strings.
412
arcfind examples
The --quiet parameter has no effect if you specify the --verbose

parameter.
If you include both the --quiet and --logfile parameters in an arcfind

command, arcfind creates an empty log file. For information on time-related parameter interactions, see Using multiple time specifications on page 4-10.
arcfind examples
This section presents examples of the arcfind command. For examples that show the use of arcfind in conjunction with arcmv, see Workflow examples on page 3-7. Example 1 Heres the sample command:
Windows: arcfind --ctime "+2010-8-30 11:59:59" --ctime -2010-10-01 --outfile file-list.txt --logfile find-log.log --exclude .tmp pdfs Unix: arcfind --ctime "+2010-8-30 11:59:59" --ctime -2010-10-01 --outfile file_list --logfile find_log --exclude _tmp pdfs
This command tells arcfind to:
Search for files starting in the pdfs directory, which is in the current
working directory
Use these criteria for the search:
The metadata for the file was last modified during the month of September, 2010. The file name doesnt include the string .tmp (Windows) or _tmp (Unix).
Write the output list to a file named file-list.txt (Windows) or file_list (Unix)
in the current working directory
Write the log to a file named find-log.log (Windows) or find_log (Unix) in

the current working directory
413
arcfind examples
Example 2 Heres the sample command:

Windows: arcfind --dirs --maxdepth 4 --prune confidential --quiet Corporate\Employees Corporate\Consultants Unix: arcfind --dirs -L --maxdepth 4 --prune confidential --Prune Confidential --quiet corporate/employees corporate/consultants
Search for directories starting in both the Corporate\Employees (Windows)

or corporate/employees (Unix) and Corporate\Consultants (Windows) or corporate/consultants (Unix) directories
Follow symbolic links (Windows does this automatically) End each directory search four levels down from the Employees or
Consultants directory
Omit directories with names that include either confidential or

Confidential (text strings in Unix are case sensitive, so the arcfind command for Unix has two --prune parameters, one for the lowercase first letter and one for the uppercase)
Not write any log information Write the output list to stdout (by default)
Windows: arcfind --minsize 50000 -o LargeFiles.txt -ve LargeFiles.log Employees\Claims Unix: arcfind --minsize 50000 -o large_files_out -ve large_files_log employees/claims
Search for files starting in the Employees\Claims (Windows) or employees/

claims (Unix) directory
Search only for files that are larger than 50,000 bytes
414
arcfind examples
Write the output list to a file named LargeFiles.txt (Windows) or large_files_

out (Unix) in the current working directory
Log detailed processing information to a file named LargeFiles.log

(Windows) or large_files_log (Unix) in the current working directory Example 4 Heres the sample command:
Windows: arcfind --ctime "+2009-12-31 11:59:59" --ctime -2010-01-01 --outfile "C:\HCP Files\Benefits-2010.txt" --logfile "C:\HCP Files\Ben-2010.log" X:\CorporateHR\Benefits Unix: arcfind --ctime "+2009-12-31 11:59:59" --ctime -2011-01-01 --outfile hcp_files/benefits_2010 --logfile hcp_files/ben_2010_log datamount/CorporateHR/Benefits
Search for files in the default namespace, starting in the CorporateHR/

Benefits directory (assuming the fcfs_data directory is mapped to the X: drive (Windows) or mounted on datamount (Unix))
Search only for files that were stored in 2010 Write the output list to a file named Benefits-2010.txt (Windows) or
benefits_2010 (Unix) in the HCP Files (Windows) or hcp_files (Unix) subdirectory of the current working directory
Log diagnostics and statistics to a file named Ben-2010.log (Windows) or

ben_2010_log (Unix) in the HCP Files (Windows) or hcp_files (Unix)
subdirectory of the current working directory
415
arcfind examples
416
5
Moving files (arcmv)
The arcmv tool lets you copy or move files. The file source or destination can be a local or remote file system or a namespace, including the default namespace. In the arcmv command line, you tell arcmv which files to copy and where to put them. This chapter presents instructions for using arcmv.
Moving files (arcmv) Using the HCP Client Tools
51
About arcmv
About arcmv
The arcmv tool copies files from one location to another. By spawning multiple threads, arcmv copies or moves large numbers of files more efficiently than other copy or move tools. The source location for arcmv (that is, where the files are copied from) can be a local or remote file system or a namespace. The target location (that is, where the files are copied to) can be a local or remote file system or a namespace. However, if you copy or move files between two HCP namespaces, both namespaces must support identical login credentials. The default action for arcmv is copy. However, you can include the --rm parameter in the arcmv command line to remove the source files after copying them, effectively changing the action to move.
arcmv tries to write each source file to its target location. If a file with the
same name already exists at that location, the write fails. The exception to this is when the target location is in an HCP namespace with versioning enabled. In this case, the write creates a new version of the file. If the arcmv command line includes the --overwrite parameter, arcmv tries to delete the existing file before writing the new one. If the delete succeeds, the write replaces the deleted file with the source file. The exception to this is when the target location is in an HCP namespace with versioning enabled. In this case, if the delete succeeds, the write creates a new version of the file. Note: A namespace can be configured to prohibit reads, writes and/or deletes. If reads or writes are prohibited, HCP returns an error for each file arcmv tries read from or write to write to the namespace. If deletes are prohibited, arcmv cannot overwrite existing files. When the target location is a namespace and the namespace access protocol is HTTP, you can use the --md parameter to provide certain system metadata values for the files being copied. arcmv applies these values as it copies the files. When the target location is a namespace and the namespace access protocol is HTTP or WebDAV, you can provide a file containing custom metadata along with each source file. If the custom metadata is well-formed XML, arcmv stores it in the namespace along with the source file.
52
About arcmv
You can include parameters in the arcmv command line that limit the number of times arcmv can retry a copy or move operation, as well as the number of errors that can occur or the number of seconds that can elapse before arcmv terminates. You can include multiple source and target locations in an arcmv command. For information on how arcmv handles multiple locations, see Using single and multiple sources and targets on page 5-19.
arcmv input
Input to arcmv can be either a text file containing a list of any number of files or, alternatively, file names you enter interactively in a Windows command-prompt window or a Unix shell. In the arcmv command, you specify the location of the files to be copied by providing one or more URLs or absolute directory paths and, optionally, a relative path. For information on how arcmv constructs the source and target locations from your specifications, see Source and target file locations on page 5-17. To store custom metadata for a file, you include the name of the file containing the custom metadata in the same line as the source file name. By default, you use a comma to separate the two file names. However, you can override this default. You would do this, for example, if your file names contain commas. The custom metadata option is available only when the source files are listed in an input file, not when you supply them interactively. In the arcmv command, you specify the source location for custom metadata files separately from the source location for the files to be copied. This location must be a directory in a local or remote file system. You use the --cmdir parameter to specify this directory. If the --cmdir parameter is not present, arcmv assumes that each entire line in the input file identifies a single source file.
53
About arcmv
Heres an example of an input file that includes custom metadata for three of four source files:
Ramirez_John_2010-10--24, Ramirez\Projects.xml Wong_Lee_2010-9-18 ExampleConsultants\Smith_Mary_2010-10-11, msmith-projects ExampleConsultants\Gold_Susan_2010-10-11, sgold-projects
Note: A namespace can be configured to disallow the addition of custom metadata for files under retention. If the target namespace is configured this way and a source file would end up being under retention (either through the --md parameter or through inheritance), arcmv doesnt store the custom metadata for that file. Important: If you plan to pipe the output from the arcfind tool directly into arcmv, be sure to read arcfind output on page 4-3, Using arcfind with arcmv on page 5-21, and Source and target file locations on page 5-17 before you construct each command.
arcmv output
Output from arcmv is optional and consists of a list of the source files that were successfully copied. If arcmv successfully stored custom metadata for a source file, the output also lists the custom metadata file on the same line. The delimiter between the two file names is the same as the one in the input file. The output list shows each source file with its full path. For example, if the source location is C:\Corporate\Employees, the base directory is 2193_John_Doe, and the file name is status_2010_11_23, the listing in the output file would be:
C:\Corporate\Employees\2193_John_Doe\status_2010_11_23
If arcmv also copies or moves custom metadata for the file where, the --cmdir parameter specifies cust-metadata and the custom metadata file name in the input file is JohnDoe\Projects.xml, the listing in the output file would be:
C:\Corporate\Employees\2193_John_Doe\status_2010_11_23, cust-metadata\ JohnDoe\Projects.xml
54
About arcmv
arcmv log
In addition to the output list, arcmv generates a log containing statistics, diagnostics, and other processing information. You can specify whether the log is written to a file or displayed on-screen, as well as which types of information it contains. Tip: In Windows, use WordPad rather than Notepad to open arcmv log files. In WordPad, each listed file appears in a separate line. In Notepad, the files are listed in a single line. The statistics in the arcmv log are labeled and formatted for easy search and script postprocessing. The labels are:
File_Stats arcmv writes one File_Stats line to the log for each file it
processes. Each line has this format:
datetime message-level File_Stats: file-order outcome file-name size file-size time number-of-seconds secs
For example:
Oct 06 20:01:48.405 INFO File_Stats: 1 Success "log117.txt" size 987 time 0.010 secs
In the File_Stats line:

Item Description
The date and time at which arcmv processed the file The importance of the message (SEVERE, WARNING, INFO, FINE, FINER, FINEST) The position of the file in the sequence of files arcmv processed, starting at 1 (one) An indicator of whether arcmv copied the file (SUCCESS) or not (FAILURE) The name of the file The size of the file, in KB The amount of time arcmv took to process the file, in seconds
datetime message-level file-order outcome file-name file-size number-ofseconds
Note: arcmv doesnt write File_Stats lines for the files it skips.
55
About arcmv
Session_Stat arcmv writes one Session_Stat line per minute to the

log during the session. Each line reports statistics accumulated since the beginning of the session and has this format:
datetime message-level Session_Stat: Time: hh:mm:ss Files: number-of-files-processed Good: number-of-successes Skipped: number-of-files-skipped Errors: number-of-errors Retries: number-of-retries Rates: total-number-of-bytes-copied KiB number-of-bytes-copied-per-second KiB/sec number-of-files-copied-per-second files/sec
For example:
Oct 06 20:01:49.407 INFO Session_Stat: Time: 00:00:01 Files: 2 Good: 2 Skipped: 0 Errors: 0 Retries: 0 Rates: 3.624 KiB 3.624 KiB/sec 2.00 files/sec
In the Session_Stat line:

Item Description
The date and time at which arcmv recorded these statistics. The importance of the message (SEVERE, WARNING, INFO, FINE, FINER, FINEST). The amount of time elapsed since the session began, shown in hours (hh), minutes (mm), and seconds (ss). The number of files arcmv has processed since the session began. The number of files arcmv has successfully copied since the session began. The number of files in the input list arcmv has skipped since the session began. arcmv skips files that already exist in the target location when you dont include the --overwrite parameter in the command line and versioning isnt enabled for the target namespace. The number of errors that have occurred since the session began. For information on what is considered an error, see arcmv errors on page 5-26. began.
datetime msg-level hh:mm:ss number-of-filesprocessed number-ofsuccesses number-of-filesskipped
number-of-errors
number-of-retries The number of retries that have occurred since the session total-number-ofbytes-copied
The total number of bytes in the files arcmv has successfully copied since the session began, expressed in KiB (1,024 bytes).
56
About arcmv
(Continued)
Item
Description
number-of-bytes- The average number of bytes arcmv has successfully copied per second since the session began, expressed in KiB (1,024 copied-perbytes). second number-of-files-c The average number of files arcmv has successfully copied per opied-per-second second since the session began.
Session_Stats arcmv writes one Session_Stats line to the log when

it finishes processing the files in the input list. (The Session_Stats line is a marker; it contains no data.)
arcmv with SSL security

HCP can be configured to require SSL security for namespace access through the HTTP and WebDAV protocols. When the system is configured this way, namespace URLs in arcmv commands must specify HTTPS. In response to an arcmv command that uses HTTPS, HCP presents an SSL server certificate. arcmv can respond to this certificate in either of these ways:
It can check whether the certificate is signed by a trusted authority:
If the client has access to a directory containing individual certificate files, arcmv can check those files for a certificate representing the authority that signed the HCP certificate. The certificates in the directory must be in PEM format. The directory itself must have been processed using the c_rehash utility supplied with OpenSSL. You use the --https-capath parameter in the arcmv command to specify the directory containing the certificate files. This option is not available for Windows clients.
If the client has access to a file containing a list of certificates in PEM format, arcmv can check that list for a certificate representing the authority that signed the HCP certificate. This file can be the default certificate file for the client, or it can be a file that you specify. You use the --https-cacert parameter in the arcmv command to specify the file that lists the certificates.
57
About arcmv
In either case, if arcmv finds such a certificate, it proceeds with the requested operation. If it cannot find such a certificate, the requested operation fails.
It can accept the certificate without checking whether its signed by a

trusted authority (--https-insecure parameter in the arcmv command) and then proceed with the requested operation.
arcmv with HCP namespaces

The arcmv command can copy or move files to or from HCP namespaces, subject to the limitations specified in Using HCP namespaces on page 5-20. To access an HCP namespace, you need a data access account. This account specifies:
Credentials HCP uses to verify that arcmv has permission to access the
namespace. These credentials consist of a username and password.
The types of operations that a client using the account can perform in
the namespace (permissions).
If these dont include the read permission, arcmv cannot read files from the namespace. If these dont include the write permission, arcmv cannot write files to the namespace. If they dont include the delete permission, arcmv cannot overwrite files in the namespace.
To get a data access account, see your namespace administrator. To use arcmv with an HCP namespace, you specify the data access account username and password in command parameters. You must also identify the hostname in the paths or use a --hostname parameter. For more information on these parameters, see arcmv syntax below
58
arcmv syntax
arcmv syntax
Heres the syntax for the arcmv command:
arcmv (-h|--help) |--version |([(--src source-location)...] [--srcroot source-path] (--dst destination-location)... [--dstroot destination-path] [--username user-name --password password [--hostname hostname]] [(--https-capath certificate-directory-path)| (--https-cacert certificate-file-spec)| --https-insecure] [--no-url-encode] [--md metadata-values] [--preserve-times] [--cmdir custom-metadata-path [--cmstrict]] [--delimiter "delimiter-character"] [--sjis] [--validate] [(-i|--infile) input-file-spec] [(-o|--outfile) output-file-spec] [--overwrite] [--rm] [(-r|--retry-limit) retry-count] [--error-limit error-count] [--time-limit number-of-seconds] [(-t|--threads) thread-count] [(-e|--logfile) log-file-spec] [(-l|--loglevel) log-level-number] [-q|--quiet] [-v|--verbose])
Note: You cannot use the --https-capath parameter in windows
59
arcmv parameter descriptions

The table below describes the parameters for the arcmv command.
Parameter --cmdir custom-metadata-path Description
Specifies the location of the custom metadata files named in the input file. custom-metadata-path can be an absolute path or the path relative to the current working directory. To specify the current working directory itself, use a period (.) alone. If you omit this parameter, arcmv assumes that each entire input line identifies a single source file. Note: This parameter is valid only when the target location is specified as a URL and arcmv is using the HTTP or WebDAV protocol.
--cmstrict
Tells arcmv that each source file must be accompanied by a custom metadata file. If a source file has no accompanying custom metadata file, arcmv doesnt copy that source file. Also, if arcmv cannot successfully store the custom metadata for a source file, it doesnt copy that source file. This parameter is valid only if the arcmv command includes the --cmdir parameter. If you include this parameter without the --cmdir parameter, the arcmv command fails. Note: If the --cmstrict parameter is not present and arcmv cannot store the custom metadata for a source file, the source file is copied anyway.
--delimiter "delimiter-character"
Specifies the delimiter used between source file names and custom metadata file names in the input file. delimiter-character must be a single character enclosed in double quotation marks. Valid values are printable ASCII characters in the range 0x20 through 0x7E. If you omit this parameter, the delimiter is the comma (,).
510

(Continued)
Parameter (--dst destination-location)...
Description
Specifies the target location for the files being copied. destination-location can be: A URL that identifies a directory in a namespace. The URL can use either a hostname or an IP address. If you use an IP address for an HCP namespace, you must also specify a --hostname parameter. An absolute or relative path to a directory in a local or remote file system.
This parameter is required. An arcmv command must include at least one --dst parameter and can have multiple. The arcmv command treats the locations specified by multiple --dst parameters as equivalents. For information on using this parameter, see Source and target file locations on page 5-17.
--dstroot destination-path
Adds the named relative path to the beginning of the specification of each file to be copied to the target location. For information on using this parameter, see Source and target file locations on page 5-17. Writes diagnostics and statistics to the specified file. log-file-spec identifies a text file. If the file doesnt exist, arcmv creates it. If the file already exists, arcmv overwrites it without notifying you. In either case, the directory in which the file is located must already exist. For information on specifying the log file, see File specifications and directory paths on page 2-5. If you omit this parameter, arcmv sends log information to stderr. When youre using arcmv interactively, stderr is the window youre working in.
(-e |--logfile) log-file-spec
--error-limit error-count
Tells arcmv not to start any new operations after the specified number of errors occur and to terminate after any current operations complete. As a result, some files may not be copied at all. Valid values for error-count are integers greater than or equal to 0 (zero). If you omit this parameter, arcmv doesnt terminate based on the number of errors that have occurred. For information on arcmv errors, see arcmv errors on page 5-26.
511

(Continued)
Parameter -h|--help --hostname
Description
Displays and describes the syntax for the arcmv command; then exits. The hostname for the HCP namespace to or from which you are moving or copying files, in this format:
namespace-name.tenant-name.hcp-name.domain-name
This parameter is required if you use an IP address in a source or destination URL for access to an HCP namespace. You do not use this parameter if the URL contains the hostname. This parameter is ignored for the default namespace.
--https-cacert certificate-file-spec
With HTTPS, tells arcmv to look in the specified file for an SSL certificate representing the authority that signed the certificate presented by HCP. For information on specifying the certificate file, see File specifications and directory paths on page 2-5. For more information on using this parameter, see arcmv with SSL security on page 5-7.
--https-capath certificate-directory-path
With HTTPS, tells arcmv to look in the specified directory for an SSL certificate representing the authority that signed the certificate presented by HCP. certificate-directory-path can be an absolute path or the path relative to the current working directory. To specify the current working directory itself, use a period (.) alone. This parameter is not valid in Windows. For more information on using this parameter, see arcmv with SSL security on page 5-7.
--https-insecure
With HTTPS, tells arcmv to accept the SSL certificate presented by HCP without checking whether its signed by a trusted authority. For more information on using this parameter, see arcmv with SSL security on page 5-7.
(-i|--infile) input-file-spec
Identifies the text file containing the list of files to be copied. For information on specifying the input file, see File specifications and directory paths on page 2-5. If you omit this parameter, arcmv waits for input from stdin. When youre using arcmv interactively, stdin is the window youre working in. To specify stdin explicitly in the arcmv command, use a hyphen (-) alone. For information on how arcmv concatenates the specifications of the files to be copied with the source and destination paths you specify in the arcmv command, see Constructing source and target paths on page 5-18.
512

(Continued)
Parameter (-l|--loglevel) log-level
Description
Tells arcmv how much processing information to log. Valid values for log-level are: 100 or FINEST Logs informational, warning, and error messages, along with detailed processing information 200 or FINER Logs informational, warning, and error messages, along with most of the detailed processing information 400 or FINE Logs informational, warning, and error messages, along with some of the detailed processing information 800 or INFO Logs informational, warning, and error messages (the default level) 900 or WARN Logs warning and error messages 1000 or SEVERE Logs only error messages
--md metadata-values
Stores the specified metadata values for each file copied to the target namespace. metadata-values has the form of an HTTP query containing one or more assignments of values to metadata properties, each separated from the next by an ampersand (&) (for example, shred=1&index=0). For more information on the --md parameter, see Metadata properties on page 5-22. HTTP rejects invalid query strings and returns a code indicating that an error has occurred. Tip: To avoid the possibility of the client operating system interpreting ampersands as control characters, precede each ampersand with the platform-specific escape character (for example, shred=1\&index=0). Note: This parameter is meaningful only when the target location is specified as a URL and arcmv is using the HTTP protocol. WebDAV does not support this feature.
513

(Continued)
Parameter --no-url-encode
Description
Tells arcmv not to URL-encode characters in HTTP and WebDAV source and target URLs. In each case, this applies to the entire URL, including elements specified in any --src, --srcroot, --dst, and --dstroot parameters, as well as in the input to arcmv. If you omit this parameter, arcmv URL-encodes all characters in source and target URLs. For more information on using the --no-url-encode parameter, see URL encoding on page 5-22.
(-o|--outfile) output-file-spec
Writes the list of copied files to the specified file. output-filespec identifies a text file. If the file doesnt exist, arcmv creates it. If the file already exists, arcmv overwrites it without notifying you. In either case, the directory in which the file is located must already exist. For information on specifying the output file, see File specifications and directory paths on page 2-5. If you specify a hyphen (-) alone as the value for this parameter, arcmv writes the list of copied files to stdout. When youre using arcmv interactively, stdout is the window youre working in. If you omit this parameter, arcmv doesnt list the copied files.
--overwrite
If arcmv tries to copy a file over an existing file and this parameter exists, it tries to delete the existing file and write the new copy. If arcmv is unable to delete an existing file (for example, if the retention period on the existing file doesnt allow it), it doesnt copy the file from the source location. If you omit this parameter, arcmv doesnt overwrite existing files. If a file with the same name exists, and the destination is not an HCP namespace with versioning enabled, arcmv doesnt copy the source file. Tip: To generate a list of the files that have been successfully copied, use the --outfile parameter.
--password
The password for the data access account specified by the --username parameter. This parameter is required to access an HCP namespace. This parameter is ignored for the default namespace.
514

(Continued)
Parameter --preserve-times
Description
Preserves the POSIX atime and mtime values of each file copied to the target namespace. If you omit this parameter, the atime and mtime values are both set to the time the file is copied to the namespace. Note: This parameter is meaningful only when the target location is specified as a URL and arcmv is using the HTTP protocol. WebDAV does not support this feature.
-q|--quiet -r|--retry-limit retry-count
Suppresses the interactive display of the session log. Tells arcmv how many times to try again to copy a file if the action fails due to a nonfatal error. Valid values for retry-count are integers greater than or equal to 0 (zero). If you omit this parameter, arcmv doesnt try again to copy files after failures. Note: arcmv does not necessarily use the same --src and --dst parameters for each subsequent attempt.
--rm
Deletes each file from the source location after its successfully copied, in effect, changing the copy action to move. If you omit this parameter, arcmv doesnt delete any source files. Note: The --rm parameter applies only to source files and not to custom metadata files.
--sjis
Tells arcmv to check each source file specification in the input file for Shift JIS encoding. If arcmv encounters a file specification thats encoded using Shift JIS, it converts the specification to UTF-8 and continues processing. arcmv does not check custom metadata file specifications for Shift JIS encoding.
515

(Continued)
Parameter (--src source-location)...
Description
Specifies the location of the files to be copied. source-location can be: A URL that identifies a directory in a namespace. The URL can use either a hostname or an IP address. If you use an IP address for an HCP namespace, you must also specify a --hostname parameter. An absolute or relative path to a directory in a local or remote file system.
An arcmv command can have multiple --src parameters; arcmv treats the locations they specify as equivalents. When the arcmv source is an HTTP location, the --src parameter is required. For information on using this parameter, see Source and target file locations on page 5-17.
--srcroot source-path
Adds the named path to the beginning of each file specification in the input list. For information on using this parameter, see Source and target file locations on page 5-17. Limits the number of concurrent arcmv threads to the specified number. Valid values for thread-count are integers in the range 1 (one) through the system-specific maximum threads per process. If you omit this value, arcmv starts ten threads that run concurrently.
-t|--threads thread-count
--time-limit number-of-seconds
Tells arcmv not to start any new operations after the specified number of seconds have passed and to terminate after any current operations complete. As a result, some files may not be copied at all. Valid values for number-of-seconds are integers greater than or equal to 0 (zero). If you omit this parameter, arcmv doesnt terminate based on a time limit.
--username
The username for the data access account youre using to copy or move files to or from an HCP namespace. This parameter is required to access an HCP namespace. This parameter is ignored for the default namespace.
-v|--verbose
Writes statistics for each file processed to the log file, if specified, or stderr, if not.
516
arcmv command considerations

(Continued)
Parameter --validate
Description
Tells arcmv to validate each source file and custom metadata file it writes to the namespace. arcmv does this by generating a cryptographic hash value for the file and comparing this value to the cryptographic hash value HCP returns for the file. Note: This parameter is valid only when the target location is specified as a URL and arcmv is using the HTTP protocol.
--version
Displays the version number of the currently installed arcmv tool; then exits.

The following sections present considerations for using the arcmv command.
Source and target file locations

The arcmv tool constructs source and target locations for the source files in an input list from the values you supply in the --src, --srcroot, --dst, and --dstroot parameters. For details of how arcmv combines the values, see Constructing source and target paths on page 5-18. These rules describe how you use these parameters to construct locations:
A --src or --dst parameter identifies the base part of a location and is

either a URL or directory path. For information on specifying multiple servers, see Using single and multiple sources and targets below.
A --src or --dst parameter directory path can be absolute or relative to

the working directory. In many circumstances, however, an absolute path is required for proper path formation.
Each arcmv command must include at least one --dst parameter. The --src parameter is optional, but may be required to generate valid
input paths. Without it, arcmv treats relative paths in the input list as starting in the current working directory, which may not be correct. Also, using the --src parameter allows the arcmv output file to show the full directory path for each copied file.
517
An arcmv command has at most one of each of the --srcroot and

--dstroot parameters. These parameters specify paths that are
relative to the source or destination location. Note: If the source and destination in an arcmv command have different paths but identify the same file (for example, if the paths differ only by IP address), using the --overwrite option results in the file being deleted. This happens because, to implement the --overwrite option, arcmv first deletes the target file and then copies the source file. In this case, because the source and target files are the same, the source no longer exists after arcmv deletes the target.
Constructing source and target paths

To locate a file to be copied, arcmv concatenates the --src value (if any), the --srcroot value (if any) and the file specification in the input file:
[src + \ +] [srcroot + \ +] file-specification
When constructing a target location, arcmv concatenates the --dst value, the --dstroot value (if any) and the file specification in the input file:
dst + \ + [ dstroot + \ +] file-specification
If the resulting paths are not absolute, they are relative to the current working directory. If any directories in the destination file paths dont exist, arcmv automatically creates them. When constructing paths, arcmv uses the elements exactly as you specify them. If a relative path includes the two periods that indicate the parent directory, arcmv puts those two periods in the file path, which results in an invalid path; for example:
http://finance.europe.hcp.example.com/rest + ..\Corporate\Consultants\ CurrentContracts + Ramirez_John_2010-10-14
becomes:
http://finance.europe.hcp.example.com/rest/../Corporate/Consultants/ CurrentContracts/Ramirez_John_2010-10-14
518
Using single and multiple sources and targets

When copying files, you can specify an HCP system in any of these ways:
Use the DNS name of the HCP system in a single --src or --dst
parameter. For example:
--dst http://www.hcp.example.com/rest
In this case, HCP selects the IP address for each copy operation, using a round-robin technique to select the address to use for each operation. HCP also ensures that each request goes to an available address. However, this technique does not allow the client to cache the DNS results.
Use multiple --src or --dst parameters with different IP addresses for

the HCP system. For example:
--dst http://192.168.1.1/rest --dst http://192.168.1.2/rest --dst http://192.168.1.3/rest
In this case, arcmv uses the specified locations in a straight round-robin; that is, it uses the first IP address for the first copy operation, the second IP address for the second operation, and so forth, starting again with the first IP address after it has run through the list. This manual round-robin approach does not ensure that the request is made to an available server. If a server is unavailable, an error occurs when arcmv tries to access the unavailable server.
Use multiple arcmv commands, each using at most one --src

parameter and one --dst parameter, and have each command use different IP addresses from the others. This technique gives you fine-grained control over the distribution of work among the servers but does not prevent the command from sending a request to an unavailable address.
519
Example of location construction

Heres an example of how target location construction works in copy operations:
In the arcmv command:

--dst http://192.168.1.1/rest --dst http://192.168.1.2/rest --dst http://192.168.1.3/rest --dstroot Corporate\Consultants\CurrentContracts
In the input file:

Ramirez_John_2010-03--24 Wong_Lee_2010-10-18 ExampleConsultants\Smith_Mary_2010-08-11 ExampleConsultants\Gold_Susan_2010-08-11
In the output file (assuming arcmv successfully copies all the files):
http://192.168.1.1/rest/Corporate/Consultants/CurrentContracts/ Ramirez_John_2010-03--24 http://192.168.1.2/rest/Corporate/Consultants/CurrentContracts/ Wong_Lee_2010-10-18 http://192.168.1.3/rest/Corporate/Consultants/CurrentContracts/ ExampleConsultants/Smith_Mary_2010-08-11 http://192.168.1.1/rest/Corporate/Consultants/CurrentContracts/ ExampleConsultants/Gold_Susan_2010-08-11
Using HCP namespaces

You can use an HCP namespace as a source, a destination, or both. However, because HCP namespaces require authentication, several requirements apply when you use arcmv to access them. Username and password requirements To access an HCP namespace, you must provide a username and password. Also, arcmv supports only a single username. Therefore:
You must specify a --username and a --password parameter for any

arcmv command that accesses an HCP namespace.
If you copy or move data between two HCP namespaces, both

namespaces must support the same username and password.
520
Host identification requirements To access an HCP namespace, the HTTP requests generated by the arcmv command must identify the namespace and tenant. Also, arcmv supports only a single --hostname parameter. Therefore:
If the HCP namespace paths start with the namespace and tenant
names, you do not need to specify a --hostname parameter.
If the HCP namespace paths start with an IP address, you must specify
a --hostname parameter.
If you are copying or moving files between two HCP namespaces, you
cannot specify IP addresses for both the source and destination.
Using arcfind with arcmv

When you use arcfind output as input to an arcmv command, arcmv constructs the source and destination paths by appending the arcfind output to the paths specified by the arcmv parameters that specify source and destination locations. You need to ensure that the combination of these paths are valid. Here are some techniques that can help ensure that the file paths constructed by arcmv are valid:
If you use a --src or --srcroot parameter in the arcmv command, do

not use absolute paths in the arcfind command.
If you use relative paths in the arcfind command, ensure that any
arcmv command --src or --srcroot parameters specify the arcfind command working directory as the source directory.
If you use absolute paths in the arcfind command, ensure that the
resulting destination paths are valid and what you intended.
If you use arcfind output as arcmv input in a single script or in scripts

that execute in the same working directory, you may not need to use the arcmv --src or --srcroot parameters. For an example that does not have --src or --srcroot parameters, see Unix example on page 3-8.
521
For more information about arcfind output as arcmv input, see arcfind output as arcmv input on page 4-11. For an example that uses arcfind output directly as input to an arcmv command, see Workflow examples on page 3-7. For more information on how arcmv constructs directory paths, see Source and target file locations on page 5-17.
Namespace access protocol considerations

These considerations apply to selecting the protocol to use in arcmv:
If the source or target for an arcmv operation is default namespace, you

should use HTTP or WebDAV as the access protocol. This is because HTTP and WebDAV provide the fastest performance.
HTTP is the only available access protocol for destinations in HCP

namespaces. (arcmv cannot use HCP namespaces as sources.)
If you use CIFS or NFS to copy files to or from a default namespace,

arcmv logs this warning message: Use the HTTP or WebDAV protocol for best performance when accessing HCP.
URL encoding
HTTP and WebDAV require certain characters, such as spaces and plus signs (+), to be percent encoded. By default, arcmv performs this encoding on all characters in HTTP and WebDAV URLs. If directory or file names in the arcmv input or --src, --srcroot, --dst, and --dstroot parameters are already encoded, performing this encoding again changes those URLs, thereby causing unexpected results. This is particularly likely to be an issue when directory and file names include non-Latin characters. To prevent arcmv from percent-encoding URLs, include the --no-url-encode parameter in the command.
Metadata properties
The metadata properties that you can specify depend on the namespace type. The default namespace uses different properties from HCP namespaces.
522
HCP namespace metadata properties

The table below describes the metadata properties you can include in the --md parameter in an arcmv command that stores files in an HCP namespace.
Property retention Description
Specifies how long the file must remain in the namespace, in the form of a retention setting that specifies a retention class or value Specifies whether the file is on hold Specifies whether the file is shredded when its deleted Specifies whether the file should be included in the search index
hold shred index
For information on the values you can specify for these metadata properties, see Using a Namespace.
Default namespace metadata properties

The table below describes the metadata properties you can include in the --md parameter in an arcmv command that stores files in a default namespace.
Property uid gid Description
The user ID of the file owner The ID of the owning group for the file
file_permissions The permissions that apply to the file directory_ permissions retention shred index
The permissions that apply to any directories arcmv creates automatically Specifies how long the file must remain in the namespace, in the form of retention setting that specifies a retention class or value Specifies whether the file is shredded when its deleted Specifies whether the file should be included in the search index
For information on the values you can specify for these metadata properties, see Using the Default Namespace.
523
Namespace directory structure

Because of the way HCP stores files, the directory structures you create and the way you store files in them can have an impact on performance. Here are some guidelines for creating effective directory structures:
Plan your directory structures before storing files. Make sure all
namespace users are aware of these plans.
Try to balance the namespace directory tree width and depth. Avoid structures that result in a single directory getting a large amount
of traffic in a short time. For example, if you ingest files rapidly, consider structures that do not store files by date and time.
If you do store files by date and time, consider the number of files
ingested during a given period of time. For example, if you ingest several hundred files per second, you might use a directory structure / year/month/day/hour/minute/second/. If you are going to ingest just a few files per second it would be better to use a less fine- grained structure.
Follow these guidelines on directory depth and size:
Do not create directory structures that are more than 20 levels deep. Instead, create flatter directory structures. Avoid placing a large number of files (greater than 100,000) in a single directory. Instead, create multiple directories and evenly distribute the files among them.
New directories
If the directory path for a file to be copied doesnt exist in the target location, arcmv creates it and stores the file in it. With the WebDAV, CIFS, or NFS protocol, if the copy or move operation fails after arcmv has already created the directory, the new directory remains empty. With the HTTP protocol, arcmv doesnt create the directory at the target location if it cannot copy the file.
524
Optimizing performance
If the target of an arcmv operation is a namespace, spreading the processing load across the HCP system can help optimize arcmv performance. Here are some ways to do that:
Within an input file, interleave source files that will be stored in

different directories.
Batch the source files into multiple input files. Then run multiple arcmv
sessions concurrently, each with a different input file. An added benefit to this technique is that if a session fails, you can rerun it using just the source files for that session.
When running multiple concurrent arcmv sessions, specify a different IP

address for the HCP system in the --dst parameter for each session.
Use multiple --dst parameters in the arcmv command, each specifying

a different IP address for the HCP system.
Use multiple threads.

Performance may improve with up to 200 threads. Because this improvement depends on several factors (such as the number of IP addresses for the HCP system, the size of the files being copied, the current load on the system, and so forth), consider experimenting with a small set of typical files to determine the best thread count for your system before running a large job. Using more than 200 threads has no performance advantage. Notes:
On Windows XP systems, the recommended maximum number of

threads is 10.
HCP uses HTTP persistent connections, with a limit of 255

concurrent connections at any given time. If other applications are accessing the namespace while arcmv is running, you should limit the number of arcmv threads to ensure that connections remain available to those applications.
525
Retrying on failure
The --retry-limit parameter tells arcmv how many times to try again to copy a file after a nonfatal error. Nonfatal errors are often due to a momentary hardware glitch or brief interruption in network service. When arcmv retries the copy, it is likely to succeed. You should always include the --retry-limit parameter in the arcmv command. Specify a large number of retries (for example, 100) to provide the most likelihood of success.
arcmv errors
You can use the --error-limit parameter to tell arcmv to terminate after a certain number of errors occur. Events that increment the error counter include:
arcmv cannot find a source file. arcmv cannot copy a source file to the target location. The arcmv command includes the --cmstrict parameter, and any of the
following applies:
A source file is listed without a custom metadata file.

arcmv cannot find a custom metadata file.
The XML in a custom metadata file is not well-formed.

arcmv cannot store the custom metadata in the namespace.
arcmv parameter interactions

Some combinations of arcmv parameters can produce unexpected results:
The --loglevel parameter works independently of the --verbose and

--quiet parameters. If you specify a log level, arcmv logs the corresponding amount of information even if the command line contains the --quiet parameter.
To enter the names of files interactively in the window youre working

in, you need to include the --quiet parameter in the arcmv command and omit the --loglevel parameter. If you omit --quiet, arcmv logs its processing information to the window too frequently for you to enter a file name.
526
Overwriting using the same source and destination directory

If the source and destination in an arcmv command have different paths but identify the same file (for example, if the paths differ only by IP address), using the --overwrite option results in the file being deleted. This happens because, to implement the --overwrite option, arcmv first deletes target file and then copies the source file. In this case, because the source and target files are the same, after arcmv deletes the target, the source no longer exists either.
Custom metadata considerations

If arcmv is saving custom metadata, the operation may fail with an HTTP 400 (Bad Request) error in either of these cases:
The XML has a large number of different elements and attributes. In this case, try restructuring the XML to have fewer different elements and attributes. For example, try concatenating multiple element values, such as the different parts of an address, to create a new value for a single element. If you cannot restructure the XML to prevent failures, ask your namespace administrator about reconfiguring the namespace to prevent HCP from validating custom metadata XML.
A number of clients try to store custom metadata for multiple objects at

the same time. In this case, limit the number of concurrent requests from clients to the namespace.
527
arcmv examples
arcmv examples
This section presents examples of the arcmv command. For examples that show the use of arcmv in conjunction with arcfind, see Workflow examples on page 3-7. Example 1 Heres the sample command:
Windows: arcmv --src C:\MyDocs\Work --dst https://www.hcp.example.com/rest/BusDocs/Mktg --dstroot Whitepapers --infile ..\MoveWork.txt --outfile ..\MoveSuccess.txt --https-insecure --overwrite --rm -r 100 --threads 10 --time-limit 60 --loglevel 1000 Unix: arcmv --src /root/my_docs/work --dst https://www.hcp.example.com/rest/BusDocs/Mktg --dstroot whitepapers --infile ../move_work --outfile ../move_success --https-insecure --overwrite --rm -r 100 --threads 10 --time-limit 60 --loglevel 1000
This command tells arcmv to:
Use C:\MyDocs\Work (Windows) or /root/my_docs/work (Unix) as the source

location
Use https://www.hcp.example.com/rest/BusDocs/Mktg as the target location

(this identifies a directory in an HCP namespace)
Add the relative path Whitepapers (Windows) or whitepapers (Unix) to the

beginning of each file specification in the input file before copying the file to the target location
Accept the SSL server certificate presented by HCP without checking

whether its signed by a trusted authority
Use the input file named MoveWork.txt (Windows) or move_work (Unix) in

the parent directory of the current working directory
Write the list of copied files to MoveSuccess.txt (Windows) or move_success

(Unix) in the current working directory
528
arcmv examples
Delete and replace any existing file at the target location that has the
same name as a file being copied, provided the delete operation is allowed for the existing file
Delete each file from the source location after successfully copying it Retry the copy operation up to 100 times for any file for which nonfatal
errors occur
Use ten concurrent threads to perform the copy operations Not start any new copy operations after 60 seconds and terminate after
any current operations complete
Write only error messages (SEVERE) to stderr

Windows: arcmv --src C:\CorporateDiv\Finance --srcroot DailyReports\2010-10-24 --dst http://finance.europe.hcp.example.com/rest/CorporateHQ/FinancialReports --dstroot 20101024 --hostname finance.europe.hcp.example.com --username finreps --password P1Ss#rd35Z -i DRpts-2010-10-24.txt -o CopyResults-2010-10\Financials-2010-10-24-out.txt --md shred=true -r 50 --error-limit 25 -ve CopyLogs-2010-10\Financials-2010-10-24.log Unix: arcmv --src /root/corporate_div/finance --srcroot daily_reports/2010_10_24 --dst http://finance.europe.hcp.example.com/rest/CorporateHQ/FinancialReports --dstroot 20101024 --hostname finance.europe.hcp.example.com --username finreps --password P1Ss#rd35Z -i drpts_2010_10_24 -o copy_results_2010_10/financials_2010_10_24_out --md shred=true -r 50 --error-limit 25 -ve copy_logs_2010_10/financials_2010_10_24_log
Use C:\CorporateDiv\Finance (Windows) or /root/corporate_div/finance (Unix)

as the source location
Add the relative path DailyReports\2010-10-24 (Windows) or daily_reports/

2010_10_24 (Unix) to the beginning of each file specification in the input
file before looking for the file in the source location
Use http://finance.europe.hcp.example.com/rest/CorporateHQ/FinancialReports as
the target location (this identifies a directory in an HCP namespace)
Use HTTP as the namespace access protocol
529
arcmv examples
Add the relative path 20101024 to the beginning of each file specification
in the input file before copying the file to the target location
Use the input file named DRpts-2010-10-24.txt (Windows) or drpts_2010_10_

24 (Unix) in the current working directory
Write the list of copied files to CopyResults-2010-10\

Financials-2010-10-24-out.txt (Windows) or copy_results-2010-10/financials_ 2010_10_24_out (Unix) in the current working directory
Set the file shred metadata value to true Retry the copy operation up to 50 times for any file for which nonfatal
errors occur
Not start any new copy operations if more than 25 nonfatal errors occur
and terminate after any current operations complete
Include individual file statistics in the arcmv log Log both file and session statistics to CopyLogs-2010-10\
Financials-2010-10-24.log (Windows) or copy_logs_2010_10/financials_2010_10_ 24_log (Unix) in the current working directory

Windows: arcmv --src Z:\DistributionDiv\BOMs --srcroot ServiceInitiated --dst http://192.168.101.110/webdav/fcfs_data --dst http://192.168.101.112/webdav/fcfs_data --dst http://192.168.101.117/webdav/fcfs_data --dstroot DistributionDiv\BOMs\Init-Service --cmdir ServiceBOM-CM --cmstrict -i Z:\DistributionDiv\BOMs\Service-October-2010.txt -o StoredRecords\Service-October-2010-Copied.txt --rm -r 80 -v -e StoredRecords\Service-October-2010-Copied.log Unix: arcmv --src /dist_div_mt/distribution_div/BOMs --srcroot service_initiated --dst http://192.168.101.110/webdav/fcfs_data --dst http://192.168.101.112/webdav/fcfs_data --dst http://192.168.101.117/webdav/fcfs_data --dstroot distribution_div/BOMs/init_service --cmdir service_bom_cm --cmstrict -i /dist_div_mt/distribution_div/BOMs/service_october_2010 -o stored_records/service_october_2010_copied --rm -r 80 -v -e stored_records/service_october_2010_copied_log
530
arcmv examples
Use Z:\DistributionDiv\BOMs (Windows) or /dist_div_mt/distribution_div/BOMs

(Unix) as the source location
Add the directory ServiceInitiated (Windows) or service_initiated (Unix) to

the beginning of each file specification in the input file before looking for the file in the source location
Use http://192.168.101.110/webdav/fcfs_data, http://192.168.101.112/webdav/

fcfs_data, and http://192.168.101.117/webdav/fcfs_data as the target locations
in a round-robin, where these locations all point to the HCP system
Use WebDAV as the namespace access protocol Add the relative path DistributionDiv\BOMs\Init-Service (Windows) or
distribution_div/BOMs/init_service (Unix) to the beginning of each file
specification in the input file before copying the file to the target location
Look for custom metadata files in the ServiceBOM-CM (Windows) or

service_bom_cm (Unix) directory in the current working directory
Not copy any files that are not accompanied by custom metadata Use Z:\DistributionDiv\BOMs\Service-October-2010.txt (Windows) or /dist_div_
mt/distribution_div/BOMs/service_october_2010 (Unix) as the input file
Write the list of copied files to StoredRecords\Service-October-2010-Copied.txt

(Windows) or stored_records/service_october_2010_copied (Unix) in the current working directory
Delete the original files from the source location after they are
successfully copied
Retry the copy operation up to 80 times for any file for which nonfatal
errors occur
Log both file and session statistics to StoredRecords\

Service-October-2010-Copied.log (Windows) or stored_records/service_october_ 2010_copied_log (Unix) in the current working directory.
531
arcmv examples
532
6
Creating directories (arcmkdir)
The arcmkdir tool creates empty directories in a default namespace or a local or remote file system. In the arcmkdir command line, you specify one or more target locations and one or more relative paths that end with a directory that does not exist in the target locations. This chapter presents instructions for using arcmkdir. Note: You cannot use arcmkdir to create directories in HCP namespaces.
Creating directories (arcmkdir) Using the HCP Client Tools
61
About arcmkdir
About arcmkdir
The arcmkdir tool creates empty directories. The target location can be a default namespace or a local or remote file system. An arcmkdir command can include multiple target locations. For information on how arcmkdir handles multiple target locations, see arcmkdir target location processing on page 6-4.
arcmkdir constructs target locations the same way arcmv does. For
information on this, see Source and target file locations on page 5-17. Input to arcmkdir consists of one or more directory paths. arcmkdir creates any directories in each path that dont already exist. You can use arcmkdir local file systems and with WebDAV, CIFS, and NFS. You cannot use arcmkdir with HTTP. When using HTTPS with WebDAV, arcmkdir has the same options as arcmv for responding to the SSL server certificate HCP presents. For information on this, see arcmv with SSL security on page 5-7.
arcmkdir syntax
Heres the syntax for the arcmkdir command:
arcmkdir (-h|--help) |--version |((--dst destination-location)... [--dstroot destination-path] [(--https-capath certificate-directory-path)| (--https-cacert certificate-file-spec)| --https-insecure] [--no-url-encode] [-v|--verbose] new-directory-path ...)
62
arcmkdir parameter descriptions
arcmkdir parameter descriptions

The table below describes the options and argument for the arcmkdir command.
Parameter Description
Specifies the target location for the directories being created. (--dst destination-location)... destination-location can be: A URL that identifies a directory in a default namespace. The URL can use either the DNS name of the HCP system or an IP address for the system. The URL must specify a WebDAV address. An absolute or relative path to a directory in a local or remote file system. Relative paths are rooted in the working directory.
This parameter is required. An arcmkdir command must include at least one --dst parameter and can have multiple. For information on using multiple parameters, see --dst arcmkdir target location processing on page 6-4.
--dstroot destination-path Adds the named relative path to each directory path to be created in
the target location, following the directory or directories specified in the --dst parameter(s).
-h|--help --https-cacert certificate-file-spec
Displays and describes the syntax for the arcmkdir command; then exits. With HTTPS, tells arcmkdir to look in the specified file for an SSL certificate representing the authority that signed the certificate presented by HCP. For information on specifying the certificate file, see File specifications and directory paths on page 2-5. For more information on using this parameter, see arcmv with SSL security on page 5-7.
--https-capath certificate-directorypath
With HTTPS, tells arcmkdir to look in the specified directory for an SSL certificate representing the authority that signed the certificate presented by HCP. certificate-directory-path can be an absolute path or the path relative to the current working directory. To specify the current working directory itself, use a period (.) alone. You cannot use this parameter with Windows. For more information on using this parameter, see arcmv with SSL security on page 5-7.
--https-insecure
With HTTPS, tells arcmkdir to accept the SSL certificate presented by HCP without checking whether its signed by a trusted authority. For more information on using this parameter, see arcmv with SSL security on page 5-7.
63
arcmkdir command considerations

(Continued)
Parameter --no-url-encode
Description
Tells arcmkdir not to URL-encode characters in URLs. In each case, this applies to the entire URL, including elements specified in any --dst and --dstroot parameters, as well as in the new directory path. If you omit this parameter, arcmkdir URL-encodes all characters in URLs. For more information on using the --no-url-encode parameter, see URL encoding on page 5-22.
-v|--verbose --version new-directory-path ...
Writes informational, warning, and error messages and detailed processing information to stderr. Displays the version number of the currently installed arcmkdir tool; then exits. Specifies the directories arcmkdir should create. Use spaces to separate multiple directory paths.
new-directory-path is a relative path that includes one or more

directories to be created. Optionally, you can start the path with existing directories. If new-directory-path already exists in the target location, arcmkdir skips it and moves on to the next input path.
arcmkdir command considerations

The following sections present considerations for using the arcmkdir command.
Reserved directory name

The directory name .directory-metadata is reserved. You cannot create new directories with this name.
arcmkdir target location processing

You can specify an HCP system as a target in an arcmkdir command in either of these ways:
Use the DNS name of the HCP system in a single --dst parameter. For
example:
--dst http://www.hcp.example.com/webdav/fcfs_data
64
arcmkdir examples
In this case, HCP selects the IP address for each create operation, using a round-robin technique. HCP ensures that each request goes to an available server. However, this technique does not allow the client to cache the DNS results.
Use multiple --dst parameters with different IP addresses for the HCP
system. For example:
--dst http://192.168.1.1/webdav/fcfs_data --dst http://192.168.1.2/webdav/fcfs_data --dst http://192.168.1.3/webdav/fcfs_data
In this case, arcmkdir uses the specified locations in a straight roundrobin; that is, it uses the first IP address for the first create operation, the second IP address for the second operation, and so forth, starting again with the first IP address after it has run through the list. This manual round-robin approach does not ensure that the request is made to an available address.
arcmkdir examples
The section presents examples of the arcmkdir command. Example 1 Heres the sample command:
Windows: arcmkdir --dst X:\BusDocs\Mktg --dstroot Whitepapers Generic\InHouse Generic\ThirdParty Unix: arcmkdir --dst /busdocs_mnt/busdocs/mktg --dstroot whitepapers generic/inhouse generic/third_party
This command tells arcmkdir to create two or three new directories in X:\BusDocs\Mktg\Whitepapers (Windows) or /busdocs_mnt/busdocs/mktg/ whitepapers (Unix):
If a directory named Generic (Windows) or generic (Unix) already exists

in X:\BusDocs\Mktg\Whitepapers (Windows) or /busdocs_mnt/busdocs/mktg/ whitepapers (Unix), create new directories named InHouse (Windows) or inhouse (Unix) and ThirdParty (Windows) or third_party (Unix) in it.
If Generic (Windows) or generic (Unix) doesnt exist in X:\BusDocs\Mktg\

Whitepapers (Windows) or /busdocs_mnt/busdocs/mktg/whitepapers (Unix), create it. Then create InHouse (Windows) or inhouse (Unix) and ThirdParty (Windows) or third_party (Unix) in it.
65
arcmkdir examples

Windows: arcmkdir --dst http://www.hcp.example.com/webdav/fcfs_data --verbose CorporateHQ/CustomerServiceReports CorporateDiv/CustomerServiceReports Unix: arcmkdir --dst http://www.hcp.example.com/webdav/fcfs_data --verbose corporate_hq\customer_service_reports corporate_div\customer_service_reports
This command tells arcmkdir to:
Create new directories named CustomerServiceReports (Windows) or

customer_service_reports (Unix) in both the CorporateHQ (Windows) or corporate_hq (Unix) and CorporateDiv (Windows) or corporate_div (Unix)
directories
Create the CorporateHq and CorporateDiv (Windows) or corporate_hq and

corporate_div (Unix) directories if they do not exist.
Write informational, warning, and error messages and detailed

processing information to stderr
66
7
HCP client tools use cases
Hitachi Data Systems customers represent a variety of business, industry, and research environments. These customers are using HCP client tools in conjunction with HCP to address their differing data storage needs. This chapter describes two real-life use cases in which HDS customers are successfully using HCP client tools to support their data storage strategies.
HCP client tools use cases Using the HCP Client Tools
71
Automatic image storage

The customer in this use case is a nonprofit organization dedicated to researching, educating about, and exhibiting plant life. The organization maintains a large and growing collection of dried plants, as well as a large and growing collection of digital images of those plants. Researchers create new images daily. In an effort to provide more detailed information, they are also rephotographing the collection using newer, higherresolution, digital photography equipment. The existing plant images were stored on approximately 10,000 CDs. This storage method was not only physically unwieldy, but also made access to the images a difficult process. The organization needed more storage space with a single point of public access, ideally through a web site. They wanted the images to be part of a permanent record of the plant research. And they wanted new digital images to be available to the public as soon as possible. For these reasons, the organization selected Hitachi Content Platform to implement a data storage strategy.
The challenge
The challenge in this case was to develop software that would automatically store new image files on a periodic basis. The three propositions embodied in that challenge are:
The software must be able to run without human intervention. The software must be able to use time criteria in selecting files to be
stored.
The software must reliably and repeatedly perform the storage

operation as scheduled. The customer had three additional requirements:
The software could not require any changes to their existing software
and processes.
The software could not overwrite image files already existing in the
target location.
72
The software had to notify the IT operator of the outcome of the

operation. Specifically, it needed to report:
Successful completion of the operation. Warnings for all files that were selected for storage but not copied because they already existed in the target location. The applicable researcher would review these files and decide, on an individual basis, whether the old or new image was correct. Errors that occurred.
Note: The organization used a third-party data migration tool for the initial transfer of the existing image files to HCP.
The solution
The solution in this case entailed:
Setting up a temporary centralized storage area on a Windows file

server and writing new and modified image files directly to that server. This removed the need for storing the images on CD at any point.
Writing a Python script to build and issue:
The arcfind command to search this centralized storage area and select the files modified since the last time the script ran The arcmv command to copy the selected files to the default namespace in the HCP system.
Using the Windows scheduling facility to set up the script to run on a

nightly basis.
The Python script

The Python script runs on the server where the new and modified image files are stored. The input directory paths for the arcfind command identify the branches that contain the image files. These paths are stored in a DOS batch file and passed to the script as arguments, which the script then includes in the arcfind command line. For the file selection process, the Python script needs to know when it last successfully copied files to the namespace. For this reason, each time the script runs to completion, it stores a file containing the start time of the
73
Time-critical image storage
operation. Before each execution, the script looks up the timestamp of the last successful operation and uses it as the value of the --mtime parameter with the plus sign (+) prefix in the arcfind command it builds. This tells arcfind to search the input directories for all files created or modified since the last successful operation. Like the input directory paths for arcfind, the source and target locations for the arcmv command, as well as the email address to which the script should send the results of the copy operation, are passed to the script as arguments. The arcmv tool creates a log file so the Python script can parse the results and email success and failure information to the designated addresses. The name of this file is also encoded in the script. Because the arcfind and arcmv tools run one after the other with no intervening processing, the script sets up a command line in which the output from arcfind is piped directly into arcmv. Heres an example of what such a command line looks like:
..\..\ArcTools\arcfind --mtime "+2010-10-14 03:00:00" ImageBase\NorthAmerica ImageBase\SouthAmerica ImageBase\CentralAmerica ImageBase\Europe ImageBase\Asia ImageBase\Africa ImageBase\Oceania ImageBase\ArcticRegion ImageBase\Antarctica | ..\..\ArcTools\arcmv --src C:\Research\Catalog --dst http://www.hcp-hi-res.flora.org/fcfs_data/Catalog --loglevel 900 --verbose --logfile ..\ImageReplication.log

In this use case, the customer is a university-affiliated, multidisciplinary, biological research group. Researchers in this group study life forms at their most basic level. To help with their analysis, they use an extremely high-resolution, high-speed microscope that captures thousands upon thousands of images daily, working around the clock. For each item it sees, the microscope captures a very large number of images. Captured images are saved on each researchers local hard drive on a Unix platform. Because the microscope works so quickly, the drives fill up with the image data in very short amounts of time. Whenever a drive has no more space, the microscope cannot save additional images to it until someone moves the existing images to a different storage area. The customer wanted a permanent, easily scalable repository of the captured images. They selected Hitachi Content Platform to meet this need. However, they could not save images directly in an HCP namespace because HCP needed to check policies and permissions for each file. As a result, they still faced the issue of the drives filling up.
74
The challenge
The challenge in this case was to:
Enable the microscope to work on a continuous basis, without being

blocked by local drives with no space left
Move the image files to a namespace with no human intervention

required The customer had two additional requirements:
The directory structure in the namespace had to exactly match the

structures on the local drives. This requirement applied to empty directories as well as to those containing files.
The images of any given item could not be moved to the namespace
until the microscope had captured them all.
The solution
The solution to this problem was a Unix shell script that used each of the three HCP client tools. In this script, which runs at regularly scheduled intervals on each researchers computer:
The arcfind tool selects all the image files last modified within a
specified amount of time before the current time. This ensures that the resulting list of files doesnt include only a partial set of images for any given item.
The arcmv tool moves each file in the arcfind list to the default
namespace in the HCP system. Because arcmv moves the files instead of copying them, the local drive never completely fills up. In fact, it appears to have unlimited space.
The arcfind tool selects all the directories last modified within a
specified amount of time before the current time.
The arcmkdir tool creates a new directory in the namespace for each
directory in the arcfind list. Note: Because the solution entails the use of the arcmkdir tool to create directories in the namespace, the customer needed to use the default namespace.
75
The shell script

The Unix shell script constructs the command lines for the HCP client tools. Each time it executes, the script determines the value for the --mtime parameter in the arcfind command by subtracting a specified amount of time from the current date and time. The name and location for the arcfind log file are encoded in the script, as is the input directory path. The shell script sets up a command line in which the output from arcfind is piped directly into arcmv. The target location for the arcmv command is encoded in the script. The script includes the --rm parameter in the arcmv command, which tells arcmv to delete each file after successfully copying it. This, in effect, moves the file to the HCP namespace. Additionally, because of the large number of files involved, the script specifies a thread count of 50 in the arcmv command. The name and location for the arcmv log file are encoded in the script, along with the value for the --loglevel parameter. After the arcfind and arcmv commands have executed, the script uses arcfind again, this time to find any recently modified directories. Then, using the WebDAV URL to specify the target location, it constructs the arcmkdir command and executes it, piping in the input directly from arcfind. Heres an example of an arcfind/arcmv command line the script might construct:
arcfind -q -e specimens/selection_log --mtime -2010-10-14\ 19:30:00 specimens/images | arcmv --dst http://www.specimen-image.univ.edu/fcfs_data --rm --threads 20 -e specimens/arcmv_log -q -l 900
Heres an example of an arcfind/arcmkdir command line the script might construct:

arcfind --dirs -q -e directory_selection_log --mtime -2010-10-14\ 19:37:00 specimens/images | xargs -l arcmkdir --dst http://www.specimen-image.univ.edu/webdav/fcfs_data
76
Installing the HCP client tools

The HCP client tools run on a variety of client platforms. The tools come as compiled executables for some platforms. They also come as source code that you can compile to run on other platforms. To get the executables or the source code, see your tenant administrator. This chapter explains how to:
Install the client tools executables Compile the client tools source code
Appendix1
Compiled-executable platforms
Compiled-executable platforms
The HCP client tools come as compiled executables for these platforms:
Microsoft 32-bit Windows:

2003 R2 (Standard and Enterprise Server editions) 2008 R2 (Standard and Enterprise Server editions) XP Professional Vista 7
Sun Solaris 10 SPARC Red Hat Enterprise Linux ES 5 (32 bit) IBM AIX 5.3 HP-UX 11i v1 (11.11) on PA-RISC
Installing the executables

To install the client tools executables: 1. Use the appropriate tool for your client platform to unpack the downloaded installation file into the directory of your choice. 2. Add the path for the directory into which you unpacked the client tools to the dynamic library path environment variable for your client platform, as indicated in the table below.
Platform
Windows PATH Tip: With Windows, if you always either run the tools from the directory in which theyre installed or specify the directory path in the tool command, you dont need to add the path to the PATH variable. Solaris SPARC Linux LD_LIBRARY_PATH LD_LIBRARY_PATH
Variable
Appendix-2
Installing the HCP client tools Using the HCP Client Tools
Compiling the client tools source code

(Continued)
Platform
AIX HP-UX LIBPATH SHLIB_PATH
Variable

The instructions in this section are for compiling the client tools source code in a Unix environment. For help compiling the source code in a Windows environment, see your tenant administrator. Before compiling the client tools, you need to ensure that these third-party libraries are installed on your client, as well as the programs gmake and gunzip:
Libcurl Libssl Libidn Libintl Libssh2 Libiconv

To compile the downloaded client tools source code: 1. Enter these commands to unpack the downloaded installation file:
gunzip arctools-source.tgz tar xf arctools-source.tar
This creates a subdirectory named arctools-source in the current directory. 2. Change to the created subdirectory:
cd arctools-source
3. Enter this command to configure the build environment for the client tools:
./configure
Appendix-3
If any of the required third-party libraries are not installed on your system, the configuration request fails. The failure message lists the libraries that are missing. 4. Enter this command to build the client tools:
make install
This builds the client tools executables and stores them in the arctoolssource directory.
Appendix-4
Glossary
A
access protocol
See namespace access protocol.
active search system

The system that works with the enabled search facility to perform searches and return results to the HCP Search Console. The active search system also maintains an index of objects in searchable namespaces, which it uses for fast retrieval of search results.
arcfind
The HCP client tool used to generate a list of files that meet a set of specified criteria.
arcmkdir
The HCP client tool used to create empty directories in a local or remote file system or in the default namespace in an HCP system.
arcmv
The HCP client tool used to copy or move files from a location on a client computer or a namespace in an HCP system to another location on a client computer or another namespace.
atime
Metadata that initially specifies the date and time at which a file was last accessed. Users and applications can change this metadata, thereby causing it to no longer reflect the actual access time. Note: This is not the normal POSIX usage for atime.
Glossary1
CIFS
C
CIFS
Common Internet File System. One of the protocols HCP uses to provide access to the contents of the default namespace. CIFS lets Windows clients access files on a remote computer as if they were part of the local file system.
client tools
See HCP client tools.
ctime
POSIX metadata that specifies the date and time of the last change to the metadata for a file
custom metadata
One or more user-defined properties that provide descriptive information about a file in HCP. Custom metadata, which is normally specified as XML, enables future users and applications to understand and repurpose file content.
D
data access account
A set of credentials that give a user or application access to one or more HCP namespaces. For each namespace, the account specifies which operations the user or application can perform.
Data Migrator
See HCP Data Migrator (HCP-DM).
default namespace
A namespace that supports the HTTP, WebDAV, CIFS, NFS, SMTP, and NDMP protocols and does not require user authentication for data access. An HCP system can have at most one default namespace.
DNS
See domain name system (DNS).
domain
A group of computers and devices on a network that are administered as a unit.
Glossary2
HCP namespace
domain name system (DNS)

A network service that resolves domain names into IP addresses for client access.
F
fixed-content data
A digital asset ingested into HCP and preserved in its original form. Once stored, fixed-content data cannot be modified.
G
GID
Group identifier.
H
HCP
See Hitachi Content Platform (HCP).
HCP client tools

A set of utilities distributed with Hitachi Content Platform that let you copy or move data from a client computer to HCP. The client tools are arcfind, arcmv, and arcmkdir.
HCP Data Migrator (HCP-DM)

An HCP utility that can transfer data from one location to another and delete data from a location. Each location can be a local file system, an HCP namespace, a default namespace, or an HCAP 2.x archive.
HCP-DM
See HCP Data Migrator (HCP-DM).
HCP namespace
A namespace that requires user authentication for data access. An HCP system can have multiple HCP namespaces.
Glossary3
Hitachi Content Platform (HCP)
Hitachi Content Platform (HCP)

A distributed storage system designed to support large, growing repositories of fixed-content data. HCP provides a single scalable environment that can be used for archiving, business continuity, content depots, disaster recovery, e-discovery, and other services. With its support for multitenancy, HCP securely segregates data among various constituents in a shared infrastructure. Clients can use a variety of industry-standard protocols and various HCP-specific interfaces to access and manipulate files in an HCP repository.
HTTP
Hypertext Transfer Protocol. One of the protocols HCP uses to provide access to the contents of a namespace.
HTTPS
HTTP with SSL security. See HTTP and SSL.
I
index
See search index.
index setting
The property that specifies whether a file should be indexed.
M
metadata
System-generated and user-supplied information about a file.
mtime
Metadata that specifies the date and time of the last change to the file data. Users and applications can change this metadata, thereby causing it to no longer reflect the actual change time.
Glossary4
POSIX
N
namespace
A logical partition of the files stored in an HCP system. A namespace consists of a grouping of files such that the files in one namespace are not visible in any other namespace. Namespaces are configured independently of each other and, therefore, can have different properties.
namespace access protocol

A protocol that can be used to transfer data to and from namespaces in an HCP system. HCP supports the HTTP protocol for access to HCP namespaces. HCP supports these protocols for access to the default namespace: HTTP, WebDAV, CIFS, NFS, SMTP, and NDMP.
NFS
Network File System. One of the protocols HCP uses to provide access to the contents of the default namespace. NFS lets clients access files on a remote computer as if they were part of the local file system.
P
permission
One of these:
In POSIX permissions, the ability granted to the owner, the members of a group, or other users to access a file, directory, or symbolic link. A POSIX permission can be read, write, or execute. In a data access account, the granted ability to perform a specific type of operation in a given namespace.
policy
One or more settings that influence how transactions and services work on files in HCP. Such a setting can be a property of a file, such as retention, or a property of a namespace, such as versioning.
POSIX
Portable Operating System Interface for UNIX. A set of standards that define an application programming interface (API) for software designed to run under heterogeneous operating systems.
Glossary5
protocol
protocol
See namespace access protocol.
R
repository
The aggregate of the namespaces defined for an HCP system.
retention class
A named retention setting.
retention period
The period of time during which a file stored in HCP cannot be deleted.
retention setting
The property that determines the retention period for a file.
S
search facility
An interface between the search functionality provided by a system such as HDDS or HCP and the HCP Search Console. Only one search facility can be enabled at any given time.
search index
An index of the metadata and key terms in files. The active search system builds, maintains, and stores this index.
service
A background process that performs a specific function that contributes to the continuous tuning of the HCP system. In particular, services are responsible for optimizing the use of system resources and maintaining the integrity and availability of the data stored in the HCP repository.
shred setting
The property that determines whether a file will be shredded or simply removed when its deleted from HCP.
Glossary6
WebDAV
shredding
The process of deleting a file and overwriting the locations where its bytes were stored in such a way that none of its data or metadata can be reconstructed. Also called secure deletion.
SSL
Secure Sockets Layer. A key-based Internet protocol for transmitting documents through an encrypted link.
system metadata
System-managed properties that describe the content of a file. System metadata includes policies, such as retention and data protection level, that influence how transactions and services affect the file.
T
tenant
An administrative entity created for the purpose of owning and managing namespaces and data access accounts. Tenants typically correspond to customers, business units, or individuals.
U
UID
User ID.
Unix
Any UNIX-like operating system (such as UNIX itself or Linux).
V
versioning
A feature that allows the creation and management of multiple versions of a file.
W
WebDAV
Web-based Distributed Authoring and Versioning. One of the protocols HCP uses to provide access to the contents of the default namespace. WebDAV is an extension of HTTP.
Glossary7
WORM
WORM
Write once, read many. A data storage property that protects the stored data from being modified or overwritten.
Glossary8
Index
Symbols
as keyword prefix 2-4 representing stdin 5-12 representing stdout 5-14 with time parameters 4-9 -- as keyword prefix 2-4 + with time parameters 4-9 .directory_metadata 6-4 = with time parameters 4-9 syntax 4-5 time-based file selection 4-94-11 in time-critical image storage use case 7-6 arcmkdir See also HCP client tools; command lines about 2-2, 6-2 advantages 3-4 command considerations 6-46-5 examples 6-56-6 with HTTPS 6-2, 6-3 input 6-2 namespace access protocols supported 6-2 parameter descriptions 6-36-4 reserved directory name 6-4 round-robin processing 6-46-5 syntax 6-2 target locations 6-2, 6-3 in time-critical image storage use case 7-6 URL encoding 5-22 arcmv See also HCP client tools; copying files; moving files about 2-2, 5-25-3 accessing HCP namespaces with 5-8, 5-20 5-21 advantages 3-3 with arcfind 4-114-12, 5-215-22 authenticated access 5-8 command considerations 5-175-26 custom metadata 5-2 errors 5-26 examples 5-285-31 host name considerations 5-21 with HTTPS 5-75-8, 5-12 in image-file replication use case 7-37-4 input 5-35-4, 5-12 log 5-55-7 metadata 5-2, 5-13 optimizing performance 5-25
A
absolute directory paths 2-5 access protocols See namespace access protocols advantages of HCP client tools 3-33-4 of scripts 3-63-7 alternative tools 3-23-4 arcfind See also HCP client tools; command lines; finding files about 2-2, 4-2 advantages 3-3 with arcmv 4-114-12, 5-215-22 command considerations 4-94-13 examples 4-134-15 HCP as file source 4-15 how it works 4-2 in image-file replication use case 7-37-4 input directories 4-9 logs 4-34-4 multiple time specifications 4-104-11 output 4-3 parameter descriptions 4-54-9 parameter interactions 4-124-13 piping output into arcmv 3-4, 4-2 Shift JIS encoding 4-8 size-based file selection 4-7, 4-8
Index1
arcmv, output
output 5-4 parameter descriptions 5-105-16 parameter interactions 5-26 preserving atime and mtime 5-15 protocol choice 5-22 round-robin processing 5-19 Shift JIS encoding 5-15 source locations 5-25-3, 5-16 source locations, constructing 5-175-20 syntax 5-9 target locations 5-25-3, 5-11 target locations, constructing 5-175-20 in time-critical image storage use case 7-6 URL encoding 5-22 user name and password considerations 5-20 validating files 5-17 arguments 2-4 atime parameter, arcfind about 4-5 datetime format 4-10 multiple 4-104-11 with other time parameters 4-11 specifying 4-94-11 atime, preserving with arcmv 5-15 authenticated access, arcmv 5-8 See also hostname parameter, arcmv; password parameter, arcmv; username parameter, arcmv Avg/file (in arcfind logs) 4-4 command considerations arcfind 4-94-13 arcmkdir 6-46-5 arcmv 5-175-26 command conventions for HCP client tools 2-3 2-6 command lines See also HCP client tools; arcfind; arcmkdir; arcmv; parameters arguments 2-4 case sensitivity 2-4 directory paths 2-52-6 file specifications 2-52-6 HCP client tools 2-3 keyword parameters 2-3 keyword prefixes 2-4 parameters 2-32-4 separators 2-4 structure 2-4 command-line tools See HCP client tools; command lines common parameters 2-5 compiling client tools source code A-3A-4 concatenating single-letter keyword parameters 2-4 constructing source locations 5-175-20 target locations 5-175-20, 6-2 conventions HCP client tool commands 2-32-6 HCP directory structure 3-5 copying files arcmv 5-15-31 tools for 3-33-4 creating directories with arcmv 5-24 directories, tools for 3-4 empty directories with arcmkdir 6-16-6 lists of directories 4-14-15 lists of files 4-14-15 lists of files, tools for 3-43-5 criteria for finding files 3-43-5 ctime parameter, arcfind about 4-6 datetime format 4-10 examples 4-13, 4-15 multiple 4-104-11 with other time parameters 4-11 specifying 4-94-11 custom metadata See also metadata; system metadata about 1-3 in input file 5-35-4 location 5-10
C
case sensitivity commands 2-4 text strings 4-11 choosing namespace access protocols for copy or move 3-6 tools for copy or move 3-6 tools for find 3-5 CIFS protocol 2-2, 5-22 client tools compiling source code A-3A-4 installing executables A-2A-3 platforms A-2 See HCP client tools cmdir parameter, arcmv See also custom metadata about 5-10 example 5-305-31 cmstrict parameter, arcmv See also custom metadata about 5-10 example 5-305-31
Index2
files
in output listing 5-4 requiring 5-10 storing for files 5-2 arcmv 5-11 arcmv examples 5-285-29, 5-295-30, 5-305-31 constructing target locations 5-175-18
D
data See files data access accounts 5-8 dates and times, specifying 4-94-11 datetime in arcmv file statistics 5-5 in arcmv session statistics 5-6 format 4-10 parameter interactions 4-11 deciding where to put files in a namespace 3-5 default namespace See also HCP namespaces; namespaces about 1-2 metadata properties 5-23 deletions prohibited 5-2 delimiter parameter, arcmv 5-10 See also custom metadata depth of directory trees, arcfind maximum 4-7 dir_permissions metadata property 5-23 directories as arcfind input 4-9 arcfind output as arcmv input 4-114-12, 5-215-22 creating empty with arcmkdir 6-16-6 creating with arcmv 5-24 generating lists of 4-14-13, 4-14, 7-6 pruning 4-8 reserved name 6-4 specifying 2-52-6 structure in HCP 3-5, 5-24 tools for creating 3-4 Directories processed (in arcfind logs) 4-3 dirs parameter, arcfind about 4-6 examples 4-14, 7-6 with maxdepth parameter 4-12 Distribution of File Sizes (in arcfind logs) 4-4 DNS manager 5-19 dst parameter arcmkdir 6-3 arcmkdir examples 6-56-6, 7-6 arcmv 5-11 arcmv examples 5-285-29, 5-295-30, 5-305-31, 7-4, 7-6 constructing target locations 5-175-18 multiple 5-11 dstroot parameter arcmkdir 6-3 arcmkdir example 6-5
E
e parameter See also logfile parameter; logs arcfind 4-6 arcfind examples 4-144-15, 7-6 arcmv 5-11 arcmv examples 5-295-30, 5-305-31, 7-6 empty directories creating with arcmkdir 6-16-6 creating with arcmv 5-24 encoding URLs 5-22 equal signs with time parameters 4-9 error-limit parameter, arcmv 5-26 about 5-11 example 5-295-30 errors arcmv 5-26 limiting in arcmv 5-11 Errors (in arcmv session statistics) 5-6 escape character in datetime format 4-10 in text strings 4-11 examples about 2-6 arcfind 4-134-15 arcmkdir 6-56-6 arcmv 5-285-31 scripts 3-73-9 use cases 7-17-6 workflow 3-73-9 exclude parameter, arcfind about 4-6 example 4-13 multiple 4-12 text strings 4-11
F
file statistics in arcmv logs 5-5 file_permissions metadata property 5-23 File_Stats (in arcmv file statistics) 5-5 file-name (in arcmv file statistics) 5-5 file-order (in arcmv file statistics) 5-5 files See also logs copied by arcmv 5-35-4 copying 5-15-31 deciding where to put 3-5 finding 4-14-15 identifying for copy or move 3-43-5
Index3
files, moving
moving 5-15-31 overwriting 5-2, 5-14 size-based selection 4-7, 4-8 specifying 2-52-6 tools for copying 3-33-4 tools for finding 3-3 tools for moving 3-33-4 Files (in arcmv session statistics) 5-6 Files processed (in arcfind logs) 4-4 files/sec (in arcmv session statistics) 5-7 file-size (in arcmv file statistics) 5-5 -file-spec in syntax 2-5 filtered (in arcfind logs) 4-4 final delimiter in directory paths 2-5 finding files See also arcfind arcfind 4-14-15 criteria for 3-43-5 in HCP 4-2 tools for 3-3 FINE log level 5-13 FINER log level 5-13 FINEST log level 5-13 fixed-content storage system 1-2 following symbolic links 4-6 HCP client tools See also arcfind; arcmkdir; arcmv; command lines about 2-2 advantages 3-33-4 alternatives 3-23-4 arcfind 4-14-15 arcmkdir 6-16-6 arcmv 5-15-31 command conventions 2-32-6 common parameters 2-5 compared with HCP-DM 3-2 directory paths 2-52-6 file specifications 2-52-6 relationship to HCP 2-3 use cases 7-17-6 using 3-13-9 HCP Data Migrator, compared with HCP client tools 3-2 HCP namespaces See also namespaces; default namespace about 1-2 accessing using arcmv 5-8, 5-205-21 metadata properties 5-23 HCP-DM, compared with HCP client tools 3-2 help parameter about 2-5 arcfind 4-6 arcmkdir 6-3 arcmv 5-12 hh:mm:ss (in arcmv session statistics) 5-6 Hitachi Content Platform See HCP hold metadata property 5-23 hostname parameter, arcmv 5-12, 5-21 HTTP protocol about 2-2 arcfind restriction 4-2 arcmkdir restriction 6-2 with arcmv 5-22 backslash handling 2-6 preserving times 5-15 specifying metadata 5-10, 5-13 validating files 5-17 HTTPS arcmkdir 6-2, 6-3 arcmv 5-75-8, 5-12 https-cacert parameter arcmkdir 6-3 arcmv 5-12 https-capath parameter arcmkdir 6-3 arcmv 5-12
G
generating See also arcfind lists of directories 4-14-13, 4-14, 7-6 lists of files 4-14-15 lists of files, tools for 3-43-5 GiB 4-4 gid metadata property 5-23 Good (in arcmv session statistics) 5-6
H
h parameter See also help parameter arcfind 4-6 arcmkdir 6-3 arcmv 5-12 HCP about 1-11-2 as arcfind source 4-15 finding files 4-2 IP address selection for arcmv 5-19 IP address selection for directory creation 6-46-5 relationship to HCP client tools 2-3 source location processing 5-19 target location processing 5-19, 6-46-5
Index4
maxsize parameter, arcfind

https-insecure parameter arcmkdir 6-3 arcmv 5-12 arcmv example 5-28 hyphens as keyword prefixes 2-4 representing stdin 5-12 representing stdout 5-14 with time parameters 4-9 l parameter, arcmv See also loglevel parameter; logs about 5-13 example 7-6 limiting errors in arcmv 5-11 time in arcmv 5-16 link parameter, arcfind 4-6 See also L parameter lists of directories examples of generating 4-14, 7-6 generating 4-14-13 lists of files copied by arcmv 5-35-4 file specifications in 4-3 generating 3-43-5, 4-14-15 logfile parameter See also e parameter; logs about 2-5 arcfind 4-6 arcfind example 4-13, 4-15 arcmv 5-11 arcmv example 7-4 with quiet parameter in arcfind 4-13 loglevel parameter, arcmv See also l parameter, arcmv; logs about 5-13 examples 5-285-29, 7-4 in time-critical image storage use case 7-6 with quiet parameter 5-26 with verbose parameter 5-26 logs See also files; logfile parameter; loglevel parameter, arcmv; quiet parameter; verbose parameter arcfind 4-34-4 arcmv 5-55-7 parameters 2-5
I
i parameter, arcmv See also infile parameter, arcmv about 5-12 examples 5-295-30, 5-305-31 identifying files to move or copy 3-43-5 image-file replication use case 7-27-4 index metadata property 5-23 infile parameter, arcmv See also i parameter about 5-12 example 5-285-29 INFO log level 5-13 input arcmkdir 6-2 arcmv 5-35-4, 5-12 input file, arcmv custom metadata 5-35-4 delimiter 5-10 installing client tools executables A-2A-3 interactions arcfind parameters 4-124-13 arcmv parameters 5-26 introduction to HCP client tools 2-12-6 IP addresses arcmkdir round-robin processing 6-46-5 arcmv round-robin processing 5-19
K
keyword parameters about 2-3 prefixes 2-4 required 2-4 KiB 4-4 KiB/sec (in arcmv session statistics) 5-7
M
max depth (in arcfind logs) 4-3 max passing entries (in arcfind logs) 4-4 maxdepth parameter, arcfind about 4-7 with dirs parameter 4-12 example 4-14 with symbolic links 4-12 maxsize parameter, arcfind about 4-7 with other size parameters 4-12
L
L parameter, arcfind about 4-6 example 4-14
Index5
md parameter, arcmv
md parameter, arcmv See also metadata about 5-13 example 5-295-30 message-level in arcmv file statistics 5-5 in arcmv session statistics 5-6 metadata See also system metadata; custom metadata about 1-3 properties 5-225-23 specifying with arcmv 5-2, 5-13 MiB 4-4 minsize parameter, arcfind about 4-7 example 4-144-15 with other size parameters 4-12 minus signs as keyword prefixes 2-4, 2-4 representing stdin 5-12 representing stdout 5-14 with time parameters 4-9 moving files arcmv 5-15-31 rm parameter 5-2, 5-15 tools for 3-33-4 mtime parameter, arcfind about 4-7 datetime format 4-10 examples 7-4, 7-6 multiple 4-104-11 with other time parameters 4-11 specifying 4-94-11 in time-critical image storage use case 7-3 7-4, 7-6 mtime, preserving with arcmv 5-15 multiple dst parameters 5-11 exclude parameters 4-12 prune parameters 4-12 src parameters 5-16 time specifications in arcfind 4-104-11 deciding where to put files 3-5 deletions prohibited 5-2 directory structure 3-5, 5-24 writes prohibited 5-2 NFS protocol 2-2, 5-22 no-url-encode parameter arcmkdir 6-4 arcmv 5-14 URL encoding 5-22 number-of-bytes-copied-per-second (in arcmv session statistics) 5-7 number-of-errors (in arcmv session statistics) 5-6 number-of-files-copied-per-second (in arcmv session statistics) 5-7 number-of-files-processed (in arcmv session statistics) 5-6 number-of-files-skipped (in arcmv session statistics) 5-6 number-of-retries (in arcmv session statistics) 5-6 number-of-seconds (in arcmv file statistics) 5-5 number-of-successes (in arcmv session statistics) 5-6
O
o parameter See also outfile parameter arcfind 4-8 arcfind example 4-144-15 arcmv 5-14 arcmv examples 5-295-30, 5-305-31 optimizing arcmv performance 5-25 others (in arcfind logs) 4-4 outcome (in arcmv file statistics) 5-5 outfile parameter See also o parameter arcfind 4-8 arcfind example 4-13, 4-15 arcmv 5-14 arcmv example 5-285-29 output arcfind 4-3, 4-8 arcmv 5-4, 5-14 overwrite parameter, arcmv about 5-2, 5-14 example 5-285-29 overwriting files 5-2, 5-14
N
namespace access protocols See also HTTP protocol; WebDAV protocol choosing for copy or move 3-6 supported 2-2 supported by arcmkdir 6-2 namespaces See also HCP namespaces; default namespace about 1-2
P
parameters See also command lines about 2-32-4
Index6
shred metadata property

arcfind 4-54-9 arcfind interactions 4-124-13 arcmkdir 6-36-4 arcmv 5-105-16 arcmv interactions 5-26 arguments 2-4 case sensitivity 2-4 common 2-5 keyword 2-3 keyword prefixes 2-4 logfile 2-5 passed (in arcfind logs) 4-4 password parameter, arcmv 5-14 -path in syntax 2-5 paths arcfind output as arcmv input 4-114-12, 5-215-22 See directories performance, optimizing arcmv 5-25 piping about 3-4, 4-2 examples 7-4, 7-6 platforms for client tools A-2 platforms, supported 2-3 plus signs with time parameters 4-9 POSIX metadata 1-3 prefixes for keyword parameters 2-4 preserve-times parameter, arcmv 5-15 protocols See namespace access protocols prune parameter, arcfind about 4-8 example 4-14 multiple 4-12 text strings 4-11 pruned (in arcfind logs) 4-3 Python script for image-file replication use case 7-37-4 quotation marks in datetime format 4-10 in text strings 4-11
R
r parameter, arcmv 5-15 See also retry-limit parameter, arcmv Rates (in arcmv session statistics) 5-6 relative directory paths 2-5 replication use case 7-27-4 required keyword parameters about 2-4 arcmkdir 6-3 arcmv 5-11 arcmv used with HCP namespaces 5-12, 5-14, 5-16 reserved names, .directory_metadata 6-4 retention metadata property 5-23 Retries (in arcmv session statistics) 5-6 retry-limit parameter, arcmv about 5-15 example 5-295-30 retrying on failure 5-26 rm parameter, arcmv about 5-2, 5-15 examples 5-285-29, 5-305-31, 7-6 in time-critical image storage use case 7-6 round-robin processing arcmkdir 6-46-5 arcmv 5-19
S
scripts advantages 3-63-7 examples 3-73-9 for image-file replication use case 7-37-4 for time-critical image storage use case 7-6 searching See finding secs (in arcmv file statistics) 5-5 separators in command lines 2-4 in datetime format 4-10 session statistics in arcmv logs 5-65-7 Session_Stat (in arcmv session statistics) 5-6 Session_Stats (in arcmv session statistics) 5-7 SEVERE log level 5-13 shell script for time-critical image storage use case 7-6 Shift JIS encoding arcfind 4-8 arcmv 5-15 shred metadata property 5-23
Q
q parameter See also logs; quiet parameter arcfind 4-8 arcfind examples 7-6 arcmv 5-15 arcmv example 7-6 quiet parameter See also logs; q parameter about 2-5 arcfind 4-8 arcfind example 4-14 arcmv 5-15 with loglevel parameter 5-26 with logfile parameter in arcfind 4-13
Index7
size (in arcmv file statistics)

size (in arcmv file statistics) 5-5 size parameter, arcfind about 4-8 with other size parameters 4-12 size-based file selection 4-7, 4-8 sjis parameter arcfind 4-8 arcmv 5-15 Skipped (in arcmv session statistics) 5-6 source locations arcmv 5-25-3 arcmv processing 5-19 constructing 5-175-20 HCP 4-15 specifying 5-16 spaces in datetime format 4-10 in text strings 4-11 specifying directories 2-52-6 files 2-52-6 POSIX date-time parameters 4-94-11 src parameter, arcmv about 5-16 examples 5-28, 5-295-30, 5-305-31, 7-4 multiple 5-16 source location construction 5-175-20 srcroot parameter, arcmv about 5-16 examples 5-295-30, 5-305-31 source location construction 5-175-20 SSL security arcmkdir 6-2, 6-3 arcmv 5-75-8, 5-12 statistics in arcfind logs 4-34-4 in arcmv logs 5-55-7 stdin for arcmv input 5-12 stdout for arcmv output 5-14 structure command lines 2-4 namespace directories 3-5 supported namespace access protocols 2-2 platforms 2-3 symbolic links following 4-6 with maxdepth parameter 4-12 syntax arcfind 4-5 arcmkdir 6-2 arcmv 5-9 -file-spec 2-5 -path 2-5 system metadata See also metadata; custom metadata about 1-3 specifying in arcmv 5-2
T
t parameter, arcmv 5-16 See also threads parameter, arcmv target locations arcmkdir 6-2 arcmkdir processing 6-46-5 arcmv 5-25-3 arcmv processing 5-19 constructing 5-175-20, 6-2 specifying 5-11, 6-3 tenants 1-2 text strings 4-11 threads parameter, arcmv about 5-16 examples 5-285-29, 7-6 optimizing performance 5-25 time in arcmv file statistics 5-5 in datetime parameters 4-10 limiting in arcmv 5-16 Time (in arcmv session statistics) 5-6 time specifications, multiple 4-104-11 time-critical image storage use case 7-47-6 time-limit parameter, arcmv about 5-16 example 5-285-29 times, specifying 4-94-11 tools See also HCP client tools alternative 3-23-4 choosing for copy or move 3-5, 3-6 choosing for find 3-5 for copying files 3-33-4 for creating directories 3-4 for finding files 3-3 for moving files 3-33-4 Total passed size (in arcfind logs) 4-4 total-number-of-bytes-copied (in arcmv session statistics) 5-6
U
uid metadata property 5-23 Unix arcfind examples 4-134-15 arcmkdir examples 6-56-6 arcmv examples 5-285-31
Index8
writes prohibited
shell script for time-critical image storage use case 7-6 workflow example 3-83-9 URL encoding about 5-22 arcmkdir 6-4 arcmv 5-14 use cases image-file replication 7-27-4 time-critical image storage 7-47-6 username parameter, arcmv 5-16, 5-20 using HCP client tools 3-13-9 Python script for image-file replication use case 7-37-4 viewing arcfind logs 4-3 viewing arcfind output 4-3 workflow example 3-73-8 workflow examples 3-73-9 storage 3-43-7 tasks 3-43-6 WORM 1-2 writes prohibited 5-2
V
v parameter See also logs; verbose parameter arcfind 4-8 arcfind example 4-144-15 arcmkdir 6-4 arcmv 5-16 arcmv examples 5-295-30, 5-305-31 validate parameter 5-17 verbose parameter See also logs; v parameter about 2-5 arcfind 4-8 arcmkdir 6-4 arcmkdir example 6-6 arcmv 5-16 arcmv example 7-4 with loglevel parameter 5-26 version parameter about 2-5 arcfind 4-8 arcmkdir 6-4 arcmv 5-17 viewing arcfind logs in Windows 4-3 arcfind output in Windows 4-3
W
WARNING log level 5-13 WebDAV protocol about 2-2 arcmkdir support for 6-2 with arcmv 5-22 Windows arcfind examples 4-134-15 arcmkdir examples 6-56-6 arcmv examples 5-285-31 backslashes in directory paths 2-6
Index9
Index10
Hitachi Data Systems Corporate Headquarters 750 Central Expressway Santa Clara, California 95050-2627 U.S.A. Phone: 1 408 970 1000 www.hds.com info@hds.com Asia Pacific and Americas 750 Central Expressway Santa Clara, California 95050-2627 U.S.A. Phone: 1 408 970 1000 info@hds.com Europe Headquarters Sefton Park Stoke Poges Buckinghamshire SL2 4HD United Kingdom Phone: + 44 (0)1753 618000 info.eu@hds.com
MK-96ARC005-10

UsingTheHCPClientTools PDF

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

UsingTheHCPClientTools PDF

Enviado por

Direitos autorais:

Formatos disponíveis

Hitachi Content Platform

Using the HCP Client Tools

Introduction to Hitachi Content Platform.............................................1-1

Introduction to the HCP client tools ....................................................2-1

Working with the HCP client tools ......................................................3-1

Contents Using the HCP Client Tools

.... .... .... .... .... ....

.... .... .... .... .... ....

3-4 3-4 3-5 3-6 3-6 3-7

Finding files (arcfind) ..........................................................................4-1

Moving files (arcmv) ...........................................................................5-1

Contents Using the HCP Client Tools

Creating directories (arcmkdir) ...........................................................6-1

HCP client tools use cases.................................................................7-1

Appendix: Installing the HCP client tools ....................................Appendix-1

Contents Using the HCP Client Tools

Contents Using the HCP Client Tools

Preface Using the HCP Client Tools

Chapter 4, Finding files (arcfind)

Chapter 5, Moving files (arcmv)

Preface Using the HCP Client Tools

arcfind [--dirs] input-directory-path

arcfind [(-e|--logfile) log-file-spec]

This book shows: output-file-spec You enter: CopyResults\Financials-2007-02-28-out.txt

arcfind input-directory-path ...

Preface Using the HCP Client Tools

Administering HCP This book explains how to use an HCP system to

Managing a Tenant and Its Namespaces This book contains complete

Managing the Default Tenant and Namespace This book contains

Replicating Tenants and Namespaces This book covers all aspects of

HCP Management API Reference This book contains the information

Preface Using the HCP Client Tools

Using a Namespace This book describes the properties of objects in

Searching Namespaces This book describes the HCP Seach Console.

Installing an HCP System This book provides the information you

Third-Party Licenses and Copyrights This book contains copyright

Installing an HCP 500 System Final On-site Setup This book

Installing an HCP 300 System Final On-site Setup This book

Preface Using the HCP Client Tools

Introduction to Hitachi Content Platform Using the HCP Client Tools

About Hitachi Content Platform

About Hitachi Content Platform

Namespaces and tenants

Introduction to Hitachi Content Platform Using the HCP Client Tools

Introduction to Hitachi Content Platform Using the HCP Client Tools

Introduction to Hitachi Content Platform Using the HCP Client Tools

HCP client tools overview

HCP client tools overview

The arcmkdir tool creates one or more empty directories in a local or

HTTP and WebDAV offer a simple, fast programmatic interface to

CIFS and NFS offer full file-system semantics with no programming

HCP client tools command conventions

Local file system

Remote file system

HCP client tools arcfind arcmv arcmkdir

Supported protocols HTTP/HTTPS WebDAV CIFS NFS

HCP client tools command conventions

Keyword parameters and arguments

A keyword parameter consists of a text string you enter as is (the

HCP client tools command conventions

HCP client tools command conventions

File specifications and directory paths