| OLD | NEW |
| (Empty) |
| 1 # -*- coding: utf-8 -*- | |
| 2 # Copyright 2015 Google Inc. All Rights Reserved. | |
| 3 # | |
| 4 # Licensed under the Apache License, Version 2.0 (the "License"); | |
| 5 # you may not use this file except in compliance with the License. | |
| 6 # You may obtain a copy of the License at | |
| 7 # | |
| 8 # http://www.apache.org/licenses/LICENSE-2.0 | |
| 9 # | |
| 10 # Unless required by applicable law or agreed to in writing, software | |
| 11 # distributed under the License is distributed on an "AS IS" BASIS, | |
| 12 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 13 # See the License for the specific language governing permissions and | |
| 14 # limitations under the License. | |
| 15 """Additional help about types of credentials and authentication.""" | |
| 16 | |
| 17 from __future__ import absolute_import | |
| 18 | |
| 19 from gslib.help_provider import HelpProvider | |
| 20 | |
| 21 _DETAILED_HELP_TEXT = (""" | |
| 22 <B>OVERVIEW</B> | |
| 23 This help section provides details about various precautions taken by gsutil | |
| 24 to protect data security, as well as recommendations for how customers should | |
| 25 safeguard security. | |
| 26 | |
| 27 | |
| 28 <B>TRANSPORT LAYER SECURITY</B> | |
| 29 gsutil performs all operations using transport-layer encryption (HTTPS), to | |
| 30 protect against data leakage over shared network links. This is also important | |
| 31 because gsutil uses "bearer tokens" for authentication (OAuth2) as well as for | |
| 32 resumable upload identifiers, and such tokens must be protected from being | |
| 33 eavesdropped and reused. | |
| 34 | |
| 35 gsutil also supports the older HMAC style of authentication via the XML API | |
| 36 (see "gsutil help apis"). While HMAC authentication does not use bearer | |
| 37 tokens (and thus is not subject to eavesdropping/replay attacks), it's still | |
| 38 important to encrypt data traffic. | |
| 39 | |
| 40 Prior to gsutil release 4.0 it was possible to use HTTP instead of HTTPS by | |
| 41 setting the "is_secure" configuration parameter in the [Boto] section of the | |
| 42 boto configuration file to False. However, starting with gsutil version 4.0 | |
| 43 setting is_secure to False is disallowed. For more details about different | |
| 44 credential options, see "gsutil help creds". | |
| 45 | |
| 46 | |
| 47 <B>LOCAL FILE STORAGE SECURITY</B> | |
| 48 gsutil takes a number of precautions to protect against security exploits in | |
| 49 the files it stores locally: | |
| 50 | |
| 51 - When the gsutil config command runs it sets file protection mode 600 | |
| 52 ("-rw-------") on the the .boto configuration file it generates, so only | |
| 53 the user (or superuser) can read it. This is important because these files | |
| 54 contain security-sensitive information, including credentials and proxy | |
| 55 configuration. | |
| 56 | |
| 57 - The gsutil config command also uses file protection mode 600 for the | |
| 58 private key file stored locally when you create service account | |
| 59 credentials. | |
| 60 | |
| 61 - The default level of logging output from gsutil commands does not include | |
| 62 security-sensitive information, such as OAuth2 tokens and proxy | |
| 63 configuration information. (See the "RECOMMENDED USER PRECAUTIONS" section | |
| 64 below if you increase the level of debug output, using the gsutil -D | |
| 65 option.) | |
| 66 | |
| 67 Note that protection modes are not supported on Windows, so if you | |
| 68 use gsutil on Windows we recommend using an encrypted file system and strong | |
| 69 account passwords. | |
| 70 | |
| 71 | |
| 72 <B>SECURITY-SENSITIVE FILES WRITTEN TEMPORARILY TO DISK BY GSUTIL</B> | |
| 73 gsutil buffers data in temporary files in several situations: | |
| 74 | |
| 75 - While compressing data being uploaded via gsutil cp -z, gsutil | |
| 76 buffers the data in temporary files with protection 600, which it | |
| 77 deletes after the upload is complete (similarly for downloading files | |
| 78 that were uploaded with gsutil cp -z or some other process that sets the | |
| 79 Content-Encoding to "gzip"). However, if you kill the gsutil process | |
| 80 while the upload is under way the partially written file will be left | |
| 81 in place. See the "CHANGING TEMP DIRECTORIES" section in | |
| 82 "gsutil help cp" for details of where the temporary files are written | |
| 83 and how to change the temp directory location. | |
| 84 | |
| 85 - When performing a resumable upload gsutil stores the upload ID (which, | |
| 86 as noted above, is a bearer token and thus should be safe-guarded) in a | |
| 87 file under ~/.gsutil/tracker-files with protection 600, and deletes this | |
| 88 file after the upload is complete. However, if the upload doesn't | |
| 89 complete successfully the tracker file is left in place so the resumable | |
| 90 upload can be re-attempted later. Over time it's possible to accumulate | |
| 91 these tracker files from aborted upload attempts, though resumable | |
| 92 upload IDs are only valid for 1 week, so the security risk only exists | |
| 93 for files less than that old. If you consider the risk of leaving | |
| 94 aborted upload IDs in the tracker directory too high you could modify | |
| 95 your upload scripts to delete the tracker files; or you could create a | |
| 96 cron job to clear the tracker directory periodically. | |
| 97 | |
| 98 - The gsutil rsync command stores temporary files (with protection 600) | |
| 99 containing the names, sizes, and checksums of source and destination | |
| 100 directories/buckets, which it deletes after the rsync is complete. | |
| 101 However, if you kill the gsutil process while the rsync is under way the | |
| 102 listing files will be left in place. | |
| 103 | |
| 104 Note that gsutil deletes temporary files using the standard OS unlink system | |
| 105 call, which does not perform `data wiping | |
| 106 <https://en.wikipedia.org/wiki/Data_erasure>`_. Thus, the content of such | |
| 107 temporary files can be recovered by a determined adversary. | |
| 108 | |
| 109 | |
| 110 <B>ACCESS CONTROL LISTS</B> | |
| 111 Unless you specify a different ACL (e.g., via the gsutil cp -a option), by | |
| 112 default objects written to a bucket use the default object ACL on that bucket. | |
| 113 Unless you modify that ACL (e.g., via the gsutil defacl command), by default | |
| 114 it will allow all project editors write access to the object and read/write | |
| 115 access to the object's metadata; and will allow all project viewers read | |
| 116 access to the object. | |
| 117 | |
| 118 The GCS access control system includes the ability to specify that objects are | |
| 119 publicly readable. Make sure you intend for any objects you write with this | |
| 120 permission to be public. Once "published", data on the Internet can be copied | |
| 121 to many places, so it's effectively impossible to regain read control over an | |
| 122 object written with this permission. | |
| 123 | |
| 124 The GCS access control system includes the ability to specify that buckets are | |
| 125 publicly writable. While configuring a bucket this way can be convenient for | |
| 126 various purposes, we recommend against using this permission - it can be | |
| 127 abused for distributing illegal content, viruses, and other malware, and the | |
| 128 bucket owner is legally and financially responsible for the content stored in | |
| 129 their buckets. If you need to make content available to customers who don't | |
| 130 have Google accounts consider instead using signed URLs (see | |
| 131 "gsutil help signurl"). | |
| 132 | |
| 133 | |
| 134 <B>SOFTWARE INTEGRITY AND UPDATES</B> | |
| 135 gsutil is distributed as a standalone bundle via tar and zip files stored in | |
| 136 the gs://pub bucket, as a PyPi module, and as part of the bundled Cloud | |
| 137 SDK release. Each of these distribution methods takes a variety of security | |
| 138 precautions to protect the integrity of the software. We strongly recommend | |
| 139 against getting a copy of gsutil from any other sources (such as mirror | |
| 140 sites). | |
| 141 | |
| 142 | |
| 143 <B>PROXY USAGE</B> | |
| 144 gsutil supports access via proxies, such as Squid and a number of commercial | |
| 145 products. A full description of their capabilities is beyond the scope of this | |
| 146 documentation, but proxies can be configured to support many security-related | |
| 147 functions, including virus scanning, Data Leakage Prevention, control over | |
| 148 which certificates/CA's are trusted, content type filtering, and many more | |
| 149 capabilities. Some of these features can slow or block legitimate gsutil | |
| 150 behavior. For example, virus scanning depends on decrypting file content, | |
| 151 which in turn requires that the proxy terminate the gsutil connection and | |
| 152 establish a new connection - and in some cases proxies will rewrite content in | |
| 153 ways that result in checksum validation errors and other problems. | |
| 154 | |
| 155 For details on configuring proxies see the proxy help text in your .boto | |
| 156 configuration file (generated by the gsutil config command). | |
| 157 | |
| 158 | |
| 159 <B>ENCRYPTION AT REST</B> | |
| 160 All GCS data are stored encrypted. For more information see | |
| 161 `Server-Side Encryption | |
| 162 <https://cloud.google.com/storage/docs/concepts-techniques#encryption>`_. | |
| 163 | |
| 164 | |
| 165 <B>DATA PRIVACY FROM GOOGLE EMPLOYEES</B> | |
| 166 Google employees will never look at your data unless you first explicitly | |
| 167 grant them permission to do so while troubleshooting a specific incident. | |
| 168 | |
| 169 Google will never ask you to share your credentials, password, or other | |
| 170 security-sensitive information. Beware of potential phishing scams where | |
| 171 someone attempts to impersonate Google and asks for such information. | |
| 172 | |
| 173 | |
| 174 <B>MEASUREMENT DATA</B> | |
| 175 The gsutil perfdiag command collects a variety of performance-related | |
| 176 measurements and details about your local system and network environment, for | |
| 177 use in troubleshooting performance problems. None of this information will be | |
| 178 sent to Google unless you choose to send it. | |
| 179 | |
| 180 | |
| 181 <B>RECOMMENDED USER PRECAUTIONS</B> | |
| 182 The first and foremost precaution is: Never share your credentials. Each user | |
| 183 should have distinct credentials. | |
| 184 | |
| 185 If you run gsutil -D (to generate debugging output) it will include OAuth2 | |
| 186 refresh and access tokens in the output. Make sure to redact this information | |
| 187 before sending this debug output to anyone during troubleshooting/tech support | |
| 188 interactions. | |
| 189 | |
| 190 The proxy configuration information in the .boto configuration is | |
| 191 security-sensitive, especially if your proxy setup requires user and | |
| 192 password information. Even if your proxy setup doesn't require user and | |
| 193 password, the host and port number for your proxy is often considered | |
| 194 security-sensitive. Protect access to your .boto configuration file. | |
| 195 | |
| 196 If you are using gsutil from a production environment (e.g., via a cron job | |
| 197 running on a host in your data center), use service account credentials rather | |
| 198 than individual user account credentials. These credentials were designed for | |
| 199 such use and, for example, protect you from losing access when an employee | |
| 200 leaves your company. | |
| 201 """) | |
| 202 | |
| 203 | |
| 204 class CommandOptions(HelpProvider): | |
| 205 """Additional help about security and privacy considerations using gsutil.""" | |
| 206 | |
| 207 # Help specification. See help_provider.py for documentation. | |
| 208 help_spec = HelpProvider.HelpSpec( | |
| 209 help_name='security', | |
| 210 help_name_aliases=['encryption', 'protection', 'privacy', 'proxies', | |
| 211 'proxy'], | |
| 212 help_type='additional_help', | |
| 213 help_one_line_summary='Security and Privacy Considerations', | |
| 214 help_text=_DETAILED_HELP_TEXT, | |
| 215 subcommand_help_text={}, | |
| 216 ) | |
| OLD | NEW |