Protected Environment (PE) at CHPC
This page refers to the refreshed PE that was funded partially by a NIH Shared Instrumentation Grant (1S10OD021644-01A1) Award received April 2017. The award allowed CHPC to deploy a complete refresh of the existing PE, and in the process expand the capabilities and increase the security relative to the initial CHPC PE deployment. In addition, the refreshed PE is configured to allow for expansion in a condominium fashion, in both the storage and in the HPC components. The different components of the new PE were made accessible to users as they were deployed, most during the first quarter of 2018.
CHPC operates a protected environment (PE) for researchers with sponsored research projects and work with data that is sensitive in nature, including projects involving human genomic data. These resources have been reviewed and vetted by the Information Security Office and the Compliance Office as being an appropriate place to work with Protected Health Information (PHI). If you have data that has other compliance requirements, please let us know well in advance so that we may ensure that our PE meets the requirements needed for your project. Please follow the recommendations below when requesting access to a resource in the PE.
NOTE: If you decide to work with protected &/or regulated data outside of CHPC's designated protected environment please know that you'll need to investigate any required agreement(s) that must be in place in accordance with the HIPAA privacy rule prior to creating, processing, maintaining, or transmitting ePHI/PII (protected health information &/or personally Identifiable info), such as a Business Associate Agreement or, in the case of research purposes, a valid IRB or other agreement, as appropriate. The Privacy Office contact information can be found at http://privacy.utah.edu/and IRB office at http://irb.utah.edu/ for guidelines and more information. If CHPC resources are used, we will assist you with the requirements. NOTE: A list of existing BAA's with the UofU (for those allowed via role based security) can be found via the following URL: https://pulse.utah.edu/site/comser/infpriv/Pages/Business-Associates.aspx
- Only CHPC Staff system administrators will have root to hosts in the protected environment.
- Only sponsored research projects with HIPAA/PHI or other specific data restrictions will be provisioned in the PE.
Protected Environment Resource Descriptions
- HPC cluster (redwood)
- Interactive nodes (bristlecone)
- Windows server (narwhal)
- Virtual Machine (VM) farm (prismatic/prismatic2)
- Storage
- Home -- /uufs/chpc.utah.edu/common/HIPAA/<UNID>
- Project -- /uufs/chpc.utah.edu/common/HIPAA/<project name>
- Scratch -- /scratch/general/pe-nfs1
- Archive (elm)
There is also a Protected Environment FAQ page that addresses questions that we have recieved regarding usage of the environment.
Cost of VMs in the PE
With the new VM farm, PE VMs will no longer be free. See the VM page for current pricing.
Allocations on Redwood
The use of allocations started July 1, 2018.
Unlike apexarch which was run unallocated, there will be an allocation process for time on redwood general resources. As in the general environemnt, there are two kinds of allocation:
- Quick allocation request -- 1 quarter only, 20,000 wallclock core hours
- Norrmal Allocation request -- submitted at most quarterly for up to 4 quarters at a time.
Quick allocations are for PIs who are new to having a CHPC allocation, and can be submitted at any time. The awarded time is for the remainder of the current quarter. It is expected that after gaining experience using our systems with the quick allocation, the allocation process (below) will be followed. Quick allocations are reviewed by senior CHPC staff and awarded at CHPC's discretion.
Normal allocation requests are accepted 4 times per year, according to the following schedule
- December 1st for allocation beginning January 1st
- March 1st for allocations beginning April 1st
- June 1st for allocations beginning July 1st
- September 1st for allocations beginning October 1st
As in our general environment, users in groups without allocation can run in the freecycle mode on general nodes or as guest on the owner nodes, subject to preemption by jobs with allocation or owner jobs, respectively. Allocations are currently reviewed and awards are made by CHPC staff.
As in the general environment, a group may request up to four quarters at a time such that they only need to complete this process once per year. However, should needs change, groups can re-apply at any of the quarterly request windows even if they have an existing award.
There is one request allowed per research group – if your group has multiple projects, please be sure to select all projects that will make use of redwood in the request. Requests can be made by the PI of the project or his/her delegate, provided the delegate also has a PE account. If you have the ability to complete this form, when you log into the chpc website, www.chpc.utah.edu, there will be a section under “User Roles” for “PI/Delegate in the PE”
Getting Started in the PE
Here are the steps to get started using the Protected Environment:
REDCAP: It may be that your project needs can be served by using the REDCAP (Research Electronic Data Capture) tool. REDCAP can be used to create web accessible forms, a secure database with continuous auditing, and a flexible reporting system. More information can be found at https://redcap01.brisc.utah.edu/ccts/redcap/index.php?action=training. To determine if the REDCAP tool fits with your project, require assistance or have any questions about REDCap, please contact REDCap Support or see Service Now - Survey Tools for more information.
Box: One can put PHI on the University instance of Box, http://box.utah.edu. The reference for the acceptable use of Box for PHI can be found at University of Utah Box User Agreement.pdf. Please refer to and see the section at the bottom about storage of regulated information. To use the Box instance for PHI, users need to create a specific University Box account. Personal accounts cannot be used. The UofU Box service doesn't fit the needs for all use cases; if you only need a storage space under the box amount (see https://box.utah.edu/) for current limit), then it may be a good fit. If the needs are to store and process (HTC/HPC/SQL etc) or use SAS/STATA and other applications, the needs are likely better served via CHPC’s Protected Environment (PE).
If the data is de-identified, there are no regulatory restrictions or mandates to use the Protected Environment. For more information on what information is protected, please see http://www.hhs.gov/ocr/privacy/hipaa/understanding/summary/index.html.
Step 2: Needs Assessment for CHPC, Risk assessment by governance risk & compliance office, Information sharing assessment form (ISAF) via privacy office
For NEW projects, please submit the NEEDS ASSESSMENT FORM with all required information. We will respond to your request as soon as possible. In the meantime, please feel free to CONTACT US with any questions. If you already submitted needs assessment form, please click here to see status.
Note: CHPC reviews submitted Needs Assessments weekly.
Depending upon the scope of the project, an "Inherent Risk survey" may need to be done by the Information Security Office (ISO) Governance, Risk, and Compliance (GRC) group. Information on the security policy can be found at https://regulations.utah.edu/it/4-004.php. For more informaiton aobut the process, contact this group via ISO-GRC@utah.edu. CHPC can help you with the process. A risk assessment and mitigation plan must be approved by ISPO Information Security Office. In addition, a satisfactory scan report must be completed.
If there is any information sharing with third parties, an information sharing assessment for (ISAF) needs to be initiated with the UofU privacy office to determine if a Business Associates Agreement needs to be put into place. The ISAF must be completely filled out (Typed) by the requesting department and returned for evaluation to determine the need for a Business Associate Agreement or other agreement to protect the information being shared. The ISAF form can be found at Business Associates. Please refer to the UofU Information Security and Privacy Office for details and help (and keep CHPC informed). Please be sure to send CHPC any existing contracts or BAA's you may have with 3rd parties.
Finally if there are will be any accounts provided for people outside the University (e.g., from another institution), a University of Utah "NON-EMPLOYEE CONFIDENTIALITY AND NON-DISCLOSURE AGREEMENT" must be signed and sent to CHPC for our records. Please contact CHPC if there are any questions about this form and when it is needed.
PLEASE BE SURE TO READ AND FOLLOW ALL STEPS IN THIS SECTION. YOU MUST FIRST HAVE A 'GENERAL' CHPC ACCOUNT . THEN YOU CAN GET A 'PROTECTED' CHPC ACCOUNT. IF YOU ENCOUNTER ANY PROBLEMS DON'T HESITATE TO CONTACT US BY SENDING AN EMAIL TO HELPDESK@CHPC.UTAH.EDU
FOR USERS REQUESTING ACCESS TO THE UCGD OR UCGD_Collab PROJECTS: BEFORE COMPLETING THE PE ACCOUNT APPLICATION, PLEASE DISCUSS THE PROCESS WITH UCGD PERSONNEL.
1. Acquire an account for the CHPC protected environment OR associate your existing PE account with a new project.
NOTE: The project PI must have an account provisioned in the protected environment before any project user accounts can be created.
a)If you do not have an existing CHPC account, then you need to apply for a general CHPC account first via the following URL: (if you already have a CHPC general account go to 1b) https://www.chpc.utah.edu/apps/profile/account_request.php
NOTE: If you don't have a uNID, please see https://www.chpc.utah.edu/documentation/policies/1.2AccountPolicies.php
b) If you have an existing CHPC general account and need access to the CHPC PE (protected environment) then you also need to request a Protected Environment/HIPAA CHPC account https://www.chpc.utah.edu/role/user/requestPE.php NOTE: You'll need to reference your project #, ask your PI if you don't know it.
c) Please note that if the project you are requesting access to is IRB governed, you will need to be on the IRB in order to be given access to the project and its data. The PI of the IRB is the person who needs to add people to the IRB.
d) If you have an existing CHPC PE account, but need your account associated with an additional project (or projects), you will need to go to the same form, where you will see a note about already having an account and a new link for adding an additional project.
2. Complete the CHPC HIPAA security training.
You will receive an email invitation to the training, which includes the Canvas link, once you have completed the account request. Please note that you must do this even if you have already completed the UofU HIPAA training. While we understand that this will be a duplication of training for a number of our PE users, the training does not take long and can be completed in 10-15 minutes. This training will need to be renewed each year; we will send out an announcement when it is time to renew your training.
3. Get a DUO security account:
This is an additional authentication step for additional security. If you don't have a DUO security account setup already then you'll need to do the following:
a) If you don't already use DUO security, you will need to go to the following URL and register your device. Visit: https://ese.idm.utah.edu/duo-management to add your device to the campus DUO two factor authentication service.
b) Then notify CHPC that you've completed the DUO registration - then we will request UIT to affiliate your DUO account to the proper CHPC PE group(s) and we will notify you when it's complete. see CHPC's DUO software page for more information
Once you've completed the previous steps (and have been notified by CHPC that your CHPC PE account is provisioned) AND the PE resources needs (narwhal, redwood, an existing protected environment virtual machine (VM), default project directories, etc) for the project exist, you can go to step 4 for instructions of how to access the CHPC PE.
If the project requires a new VM provisioned (one of the items requested via the Needs Assessment), you will be notified when the VM has been provisioned. If you have not yet discussed the VM requirements with CHPC, we will reach out to schedule a call or meeting with you. Any new protected environment virtual machine requests must come from the PI (or co PI). See 1.4.2 Virtual Machine Allocation Policy For VM "block" sizes and cost please refer to: https://www.chpc.utah.edu/resources/virtualmachines.php#pvf
In addiition, if the new project needs additional storage, you can submit a ticket to purchase additional project space on mamoth. See the PE storage page for details of the cost structure of storage.
Logging into the PE
1. Using an SSH client:
This is the easiest one to test before attempting to use the fastx or rdp methods below (if SSH doesn't work don't bother trying the fastx or rdp methods - contact us for help) NOTE: We have verified linux openssh client and windows putty ssh client.
-
- SSH to redwood.chpc.utah.edu (this does a DNS round robin to redwood1.chpc.utah.edu
& redwood2.chpc.utah.edu), using your campus uNID. You may want to use 'ssh -Y uNID@redwood.chpc.utah.edu' so any xwindow apps will work
(or if using windows/mac make sure to config your ssh client session to enable X11 forwarding).
- Enter your campus uNID and password then after a successful 1st authentication you'll select one of the three options that DUO two-factor presents you (for example select #1 for DUO push) then accept & confirm from your phone and you should have successfully logged in
- SSH to redwood.chpc.utah.edu (this does a DNS round robin to redwood1.chpc.utah.edu
& redwood2.chpc.utah.edu), using your campus uNID. You may want to use 'ssh -Y uNID@redwood.chpc.utah.edu' so any xwindow apps will work
(or if using windows/mac make sure to config your ssh client session to enable X11 forwarding).
If you're not familiar with SSH - here is a link with more info.
2. Using FastX to get to redwood1.chpc.utah.edu (or other linux hosts as needed)
FastX version 3 Instructions: Please see fastx3 documentation using host name you want to reach. The options available for all users are redwood1 and redwood2, e.g, redwood1.chpc.utah.edu:3300 and redwood2.chpc.utah.edu:3300 , or bristlecone1 and bristlecone2 (bristlecone1.chpc.utah.edu:3300 and bristlecone2.chpc.utah.edu:3300 ) unless you have another designated owner node).
3. Using Open OnDemand
Open OnDemand is a web portal that provides access to CHPC file systems. In the PE, OnDemand can be accessed at https://pe-ondemand.chpc.utah.edu. This is the best option for use of GUI based applicaitons such as Matlab, Jupyter notebooks, RStudio Server.
4. Using remote desktop (aka RDP) to get to the windows host narwhal.chpc.uah.edu:
-
- Use a remote desktop client on your platform to open a session to narwhal.chpc.utah.edu (use ad\uNID for the username), It may present you with a certificate if so click "yes" to accept.
- The DUO client on your phone should notify you for approval and confirm the authentication of your device.
- The RDP client will then prompt you for your username and password login information.
- See this link for detailed instructions on connecting from Windows or a Mac - useful for sending users to this link
- If all the above steps are accounted for the issue connecting might be with the RDP
version of client (seems to happen with Windows 7 Home version) - have them upgrade
the RDP client.
- RDP 8.0 https://support.microsoft.com/en-us/kb/2592687 *has a prerequisite (listed on the page)
- RDP 8.1 https://support.microsoft.com/en-us/kb/2923545 *has a prerequisite (listed on the page)
If you have any problems please contact helpdesk@chpc.utah.edu
How to access your protected environments home directory & project group directories from UNIX hosts redwood1.chpc.utah.edu or redwood2.chpc.utah.edu:
For IRB project access, the full/absolute path to IRB project data is:
/uufs/chpc.utah.edu/common/PE/<name of project directory>
Or from your homedir you can do:
cd ../<name of project directory>
(Note that tab completion will not work as the project spaces are not automounted.