Troubleshooting
Problem
After applying encryption to a directory, previously existing plain-text files appear corrupt or unreadable. . In situations where new files have been added, newer files are readable whereas older files are unreadable.
Symptom
Users receive application errors when opening files with associated creator or reader applications.
.
Files newer than a certain date perform as expected whereas files older than a certain date fail to open.
Cause
In most use cases, the Guardium Data Encryption ("GDE") agent applies encryption indiscriminately. Files that reside on guarded storage media in whatever form, plain-text or encrypted, will be decrypted upon read or encrypted upon write. Encrypt/decrypt operations must be paired and like mathematical parentheses, every open parenthesis must be matched with the correct closing parenthesis.
.
If files reside on storage in plain-text form within a guard point, they will be "decrypted" by the agent, thus appearing encrypted or corrupt to the opening application. It is also possible that files have been previously encrypted and the guard has been removed. Additional mismatch combinations such as double encryption or unordered encryption with multiple keys are possible.
.
An additional cause of apparent data corruption can be found in the policy configuration. Policies utilizing selective groups (user, process, when, etc.) create sub-groups of users with different access rights and users expecting datavisibility may be falling outside of that privilege group.
Diagnosing The Problem
Definitive diagnosis can be achieved by examining the guard point status in conjunction with the state of the data files. To determine the status of a guard point, run the command:
.
# secfsd -status guard
GuardPoint Policy Type ConfigState Status Reason
---------- ------ ---- ----------- ------ ------
/gp/00 basic local guarded guarded N/A
.
In the example shown above, the path "/gp/00" is guarded with the "basic" policy.
.
If the guard point is enabled, attempt to open the files with the creator or associated reader application. If the files can be read successfully, they are encrypted on the storage media. If the files cannot be read successfully, remove the guard point and examine the "corrupted" files with the creator application or other associated reader. If the files are plain text or can be read successfully by the creator or reader application, the files are not encrypted on storage.
.
| Guard point enabled | Guard point disabled | |
| not encrypted | cannot read | can read |
| singly encrypted | can read | cannot read |
| multiple encryption/corrupt | cannot read | cannot read |
If removing or disabling the guard point is not practical, create a user set for a test user with {permit} access (no "apply key" effect) and insert that access rule as rule #1 in the policy, and test as stated above. This test user represents the "guard point disabled" case.
.
If a checksum or other signature of the plain-text file is known, the checksum or signature will match when re-computed against the file after the guardpoint has been removed.
.
Definitive analysis of policy configuration errors is beyond the scope of this document but as general guidance:
- Set "audit" on every policy rule.
- Set the log level to INFO or lower.
- Examine the access log entries. Depending up the logging configuration, access logs are usually found on the Management Console and on the host in the VMD log (/var/log/vormetric/vorvmd_root.log for Unix, C:\ProgramData\Vormetric\DataSecurityExpert\agent\logs\orvmd.log for Windows). An entry will end with the field "Effect" and some information describing the logic.
.
<date> <time> [CGP] [ERROR]
.
In the message shown above, the policy failed to match the user ("U") on rule 1 and matched ("M") on Rule 2.
.
Diagnosis of multiple encryption/decryption is similar but more challenging:
- With the guard point enabled, copy the suspect file to an unguarded location and append "-d1" to the filename.
- Disable the guard point.
- With the guard point disabled, copy the "-d1" copy into the unguarded guard point directory.
- With the guard point disabled, copy the original suspect file to an unguarded location and append "-e1" to the filename.
- Enable the guard point.
- With the guard point enabled, test the "-d1" file with the creator or associated reader application. If the application can read the file, stop. This file has been decrypted twice to restore it properly.
- With the guard point enabled, copy the "-e1" file into the guard point.
- With the guard point enabled, copy the "-d1" file to an unguarded location and rename the file to "-d2".
- Disable the guard point.
- With the guard point disabled, test the "-e1" file with the creator or associated reader application. If the application can read the file, stop. This file has been encrypted twice to restore it properly.
- Repeat until the correct level of encrypt/decrypt cycles has been identified.
Resolving The Problem
- There are multiple ways to resolve the issue.
If a backup exists that encompasses all of the affected files, it may be more efficient to restore files from the backup. Prior to restoration, verify that the backup/restore process does not fall under a security rule with an effect of {permit}. Restore the files to the existing directory with the guard point established.
.
In the absence of a comprehensive backup, files can be encrypted using either the dataxform utility or by copying them to an established guardpoint. The steps below address a mixture of files that have been encryped/decrypted zero or one time. For files that have been encrypted/decrypted multiple times, repeat these steps.
.
Preparation
If new files have been added since the guard point was established, remove those files first. Definite identification of individual files is beyond the scope of this article, although some guidelines may be followed:
- Determine the likely date that the guardpoint was established. This will appear as message ID VMD3781I ("Successfully received and implemented a new security configuration") in the VMD log file (/var/log/vormetric/vorvmd_root.log, C:\ProgramData\Vormetric\DataSecurityExpert\agent\log\vorvmd.log). This message will be separate from a cluster of messages generated by application start-up.
- Identify a file that is older than the likely guard point date and validate that this file is encrypted with the guard point established.
- Identify a file that is newer than the likely guard point date and validate that this file is plain-text with the guard point established.
- Move all files that are newer than the guard point date out of the guard point.
.
Method 1: Using "dataxform"
- Create a data transformation policy if necessary. Use the same key selection and resource sets as the current security policy.
- Remove the guard point by removing the current policy on the directory at the Management Console.
- Copy in any plain-text flies from the preparation steps listed above.
- Apply the dataxform policy to the directory and run the dataxform program.
- Upon completion, remove the dataxform policy and re-apply the production security policy.
- Dataxform sets a lock to prevent repeated passes of encryption. If dataxform must be run additional times on the same guardpoint, remove the lock by running dataxform with the "cleanup" option:
- # dataxform --cleanup --gp <path>
.
Method 2: Copy-out/Copy-in
- Remove the guard point on the current directory.
- Rename the current directory and create a new directory in its place.
- Re-establish the guard point on the newly created directory.
- Copy files from the old directory into the new directory.
- Copy in any plain-text flies from the preparation steps listed above.
Example:
$ vmssc host delgp -p policy -d /gp/00 hostname
$ mv /gp/00 /gp/00.plain; mkdir /gp/00
$ vmssc host addgp -p policy -d /gp/00 hostname
$ rsync -a /gp/00.plain/ /gp/00/
$ find /gp/00.plain -type f -exec shred -fu {} \;
NOTE: failure to shred original data after copying may leave sensitive data in the unallocated blocks of the storage media.
Was this topic helpful?
Document Information
Modified date:
16 June 2018
UID
swg21691337