CipherDB Developer’s Guide – .NET

The Crypteron CipherDB middleware is a patent pending application layer encryption framework that allows your .NET or Java application to quickly be upgraded for strong data encryption even if the underlying database technologies or infrastructure providers are insecure. In fact CipherDB reduces the trust footprint to just your application. This means the database server, database administrators and pretty much anything outside your application instance and its OS does NOT need to be trusted for information assurance.

CipherDB works with SQL Server, Azure SQL, MySQL, Amazon RDS, Oracle or pretty much any SQL other database (sharded or not) and integration is just a couple of lines of code for the entire application. We have client libraries that extend existing ORM APIs like Entity Framework or NHibernate (C# clients) or Hibernate (Java clients) to auto-magically work with the built-in key management servers which can handle billions of keys. Best of all, we preserved the familiar programming model so you and your team are productive within minutes. In fact, here is the before and after programming model:

// Before: Unprotected regular access
using (var db = new DbContext())
{
    // do work
}
// After: Crypteron protected access
using (var secDb = new CipherDBContext())
{
    // do work. Network and database see only strongly encrypted data
    // while only your app sees unencrypted data for ease of development
}

All database access points are automatically upgraded for security in a transparent manner, so no need to check every database call separately. Behind the scenes, the encryption framework takes care of encryption, decryption, key caching, key lifecycle management, tamper detection, SQL row binds, encryption key migrations and rollovers, policy enforcement, decoupling of administrative, security and development roles as well as many other features. The overhead of the entire security pipeline is imperceptible to your end users - under 1ms for your typical record so external factors like network, bandwidth usually dominate any impact of CipherDB. For a detailed article on performance, please read our blog post here.

CipherDB also enables creating isolated security partitions for multi-tenant situations as well as ACLs based policies controlling data access. This is reduced to a simple programming model like

// Crypteron protected access
using (var secDb = new CipherDBContext("MyTenant45", "ReadOnlyRole"))
{
    // access to other tenant's security paritions denied
    // write access within this partition denied
}

CipherDB helps you focus on the core of your application without being overwhelmed by the security implications of handling sensitive data in a connected environment.

Also take a look at Crypteron's CipherStor product if your application also needs to protect unstructured data like flat-files (eg: 911 audio recording, crime scene imagery etc) that will be resident on file servers (eg: Amazon S3, Azure Blob Storage or 3rd party filers) rather than database servers. CipherDB and CipherStor use the same underlying key-management system, simplifying the application domain model.

Crypteron's CipherDB and CipherStor security framework is also available in a FIPS-140-2 certified flavor and is fully compliant with NSA Suite B. It can even be used to protect data classified as military "Top Secret".

Basic Flow

The overall process is

  1. Security classification of data: High level architectural review of what your application considers secure data and non-secure data and where they live within your database schema.
  2. Column naming: Prefixing the secure columns with Secure_ either directly in SQL (new designs) or via Entity Framework remapping (existing, legacy SQL designs)
  3. Setup your App on the Crypteron Management Dashboard and passing the API AppSecret back to your app via the web.config or app.config
  4. Integrating CipherDB middleware to your .NET app.
    TIP: We also provide some simple sample applications on GitHub.

Security classification of data

This stage is code free in the sense that it's purely a planning stage. You analyze the data that your application handles within the context of your business or operations and pick specific columns in specific tables that you think are sensitive pieces of data. For example, while social security number would be considered a sensitive field in almost every application, perhaps your use case might consider 'address' as a secure field too. All such secure columns are marked as such so CipherDB can protect them during data flow monitoring. Some illustrative examples are:
1. User profile Table, SocialSecurity column
2. Patient profile Table, First Name column
3. Payments Table, CreditCardNumber column
and so on
Remember, if you aren't sure about a particular field, it's probably a good idea to flag it as secure it because attackers can be quite creative with security breaches.

NOTE: Plan for the fact that SQL columns to be secured must be of string or binary type.

TIP: Apart from column type, it's most flexible to keep them of variable length with max upper bound. For example, nvarchar(max) or varchar(max) for string or varbinary(max) for binary. Encryption can expand the final encrypted data (why?), especially when when operating on small records so using variable length with max upper bound offers architects and developers great design flexibility since modern databases don't pre-allocate or waste space in this mode.

Flag columns as 'Secure' to CipherDB

For columns you wish to mark as secure, you can either add the [Secure] attribute to the entity's property (for Entity Framework Code-first) or prefixing the string Secure_ to the Entity's property name. E.g. CreditCardNumber column => Secure_CreditCardNumber column. The actual prefix can be customized inside the CipherDB configuration settings (see the configuration section below) but we recommend something like Secure or Encrypted such that it clearly and obviously identifies such secure fields as a constant reminder to the programmer of their sensitive nature.

If you wish to rename the Entity's property, you again have two options:

Directly renaming SQL schema

For new database designs, we recommend directly renaming the SQL columns as it's a good idea for the name to reflect the type and nature of the data it contains. You can use existing SQL database administrative tools to create your table or rename columns within existing tables.

Name mapping when SQL schema can't be changed

If you have an existing database schema and are concerned about renaming database columns and it's ripple effect - don't worry. Entity framework allows a custom mapping between SQL columns and your entity properties/fields. So you could have a legacy column like PIN but you can very easily map it to Secure_PIN in your Entity Framework/CipherDB layer. Your SQL database schema remains unchanged and CipherDB automatically picks it up and encrypts all writes to that column. See below on how to do it.

TIP: If you're renaming existing members and are using a recent version of Visual Studio, it's best to use Visual Studio's refactor menu (right click member => refactor => rename) to rename members with the 'Secure_' prefix.

Entity framework code-first

In Entity framework code-first, this mapping is usually in your ModelsMapping folder. A quick example to remap is shown below:

public UserMap()
{
    // ...
    this.Property(t => t.Secure_LegacyPIN).HasColumnName("PIN");
    // ...
}

Please see MSDN documentation for more details on mapping

Entity framework database-first

This mapping is usually stored inside your EDMX file. You need to edit the EDMX mappings inside Visual Studio, an example of which is shown below:

Entity Framework Database-first remapping

Setup your App on the Dashboard

Your CipherDB powered App orchestrates the necessary work with the back-end Key management systems at Crypteron. First you need to setup your App on the Crypteron dashboard. By default we register an App for you when you first create your Crypteron account. Select the App on the dashboard and in App Details you'll find the App's Secret Key. This is the API key securing all the key management API between your App and the Crypteron Service.

NOTE: The App Secret is NOT your data encryption key(s) but you should still guard this API key.

Integrating CipherDB middleware

Get the Crypteron Agent via NuGet. You can find them all listed here. Then you want to upgrade your existing generic Entity Framework class(es) to be powered by CipherDB. This step is slightly different depending on whether you are using Entity Framework code-first approach or the database-first approach.

Entity Framework database-first

Entity framework database-first developers follow a workflow in which the Entity Framework automatically generates the Entity Framework class, say, MyAppEntities. Since it's possible that Entity Framework may auto-generate that class again later, it's best to leave that class untouched and instead add the integration code in a newer, inherited class, say, CipherDBEntities. This is shown below in the same code:

using Crypteron.CipherCore;
using Crypteron.CipherDB;

public class CipherDBEntities : MyAppEntities
{
    public CipherDBEntities(string securityPartition = null, string asRole = null,)
    {
        // Activate CipherDB, specify security context. That's it!
        Crypteron.CipherDB.Session.Create(this, securityPartition, asRole);
    }
}

Entity Framework code-first

Entity framework code-first developers follow a workflow in which they hand-write and maintain their entity Framework class, say, MyAppEntities. Due to this they can either add the integration code above directly into MyAppEntities or in an inherited class, say, CipherDBEntities. The performance is the same; it's just a matter of preference.

Now always use the CipherDB powered class (example CipherDBEntities) instead of the generic Entity Framework MyAppEntities class. The runtime usage is identical and CipherDB will manage encryption and all key management behind the scenes.

Stored Procedures and CipherDB

CipherDB works just fine with stored procedures as long as one follows the general entity framework guidelines.

  1. You must import the stored procedures into entity framework
  2. After importing, the database-first approach still requires another step; importing it from the “store” into the “model”.
    1. Double click the EDMX file
    2. Go to Model Browser (if not visible, right click an empty area of the EDMX designer and select “Model Browser”)
    3. Right click the stored procedure and choose “Add Function Import...”
    4. This moves it to the “Function Imports” section as seen below
      CipherDB Stored Procedures
  3. The inputs and outputs to the stored procedure must be “Entities”. For example, to set the output to type “Entity”, double click the stored procedure in “Function Imports” and change to Entities as shown below
    CipherDB Stored Procedures
  4. These entities are basically rows from your secure or non-secure tables.

NOTE: For getting entities that are CipherDB secured your stored procedure must return the entire entity (i.e. must return the entire row). The stored procedure cannot simply return a small subset of fields (a subset of columns) because they wouldn’t “fit” the type of the Entity (and CipherDB will not decrypt it).

Configuration settings

Crypteron Security Framework's components are configured via the .NET application’s .config file, typically the web.config file for ASP.NET web applications or the app.config file for standalone .NET application. As usual, there is a declaration in the followed by the actual configuration itself inside the <crypteronConfig> section. A short illustrative example is shown below:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <!-- Declaration only, no customization -->
    <section name="crypteronConfig" type="Crypteron.CrypteronConfig, CipherCore, Version=2017, Culture=neutral, PublicKeyToken=e0287981ec67bb47" requirePermission="false" />
  </configSections>

  <!-- Read below for valid values and customization -->
  <crypteronConfig>
    <myCrypteronAccount appSecret="Replace this with App Secret from https://my.crypteron.com" />
    <commonConfig securePrefix="Secure_" allowNullsInSecureFields="false" bindRow="true" />
    <cipherDbConfig migrationPolicy="ReadOld" />
    <debugLogs>
      <debugLogFile logFilename="MyApp-CipherDB.log" filterLevel="Warning"/>
      <debugConsole filterLevel="Information"/>
    </debugLogs>
  </crypteronConfig>
</configuration>

Detailed information for various configuration parameters are documented below.

Configuration through code

Starting with release 2015.11.2, one can also access and modify the configuration via something like:

var cfg = Crypteron.CrypteronConfig.Config;
cfg.CipherStorConfig.ReportEventEveryXBytes = 2*1024*1024*1024; // every 2GB
cfg.DebugLogs.DebugLogFile.LogFilename = "Crypteron-Debug.log";
cfg.DebugLogs.DebugLogFile.FilterLevel = TraceEventType.Information;

WARNING: Many settings are expected stay the same across the lifetime of the application process (e.g. AppSecret) or even the lifetime of the data persisted (e.g. BindRow). We therefore recommend setting the configuration at App startup.

Updates to configuration spec

The configuration specification was updated in release 2015.8. For a comparison of the older vs newer syntax, please check out this post.

<crypteronConfig>

Within the configuration XML, this is the top-level element within with all other Crypteron specific configuration exists.

<myCrypteronAccount>

To register your app with your MyCrypteron account.

Name Example Description
appSecret appSecret="EiB/xyx123abcdefghijklmnopqrstuvwxyz1234567890abcd==" This is the what you obtain from your MyCrypteron management dashboard

<commonConfig>

All common settings across the various Crypteron Security Framework modules are captured in this XML configuration element as follows:

Name Example Description
securePrefix securePrefix="Secure_" Objects or entities with properties or fields with this prefix (example: "Secure_CreditCard") will be protected by Crypteron's security pipeline. Properties or fields without this prefix (example: "ItemIdPK") will flow in bypass mode outside the Crypteron Security pipeline.Default: "Secure_"
allowNullsInSecureFields allowNullsInSecureFields="false" Setting this to "true" means plaintext as well as ciphertext values of NULL are allowed. Some developers find permitting NULLs to be more flexible or practical (e.g. SQL column NULLs) but this is less secure since an attacker may silently erase data by writing NULLs without raising any alarms. This in turn is because NULLs cannot store any metadata needed to enforce integrity protection. Effectively NULLs are passed through as-is across both the decryption and encryption operations.

Setting this to "false" is more secure since Crypteron will now throw an exception if it find NULLs prior to decryption. We suggest using an empty string or empty byte array instead of NULL when writing. This way when plaintext is empty, it's ciphertext is non-empty with helpful metadata that can enforce tamper protection checks next time decrypting it.

Default: false

bindRow bindRow="true" Row binding prevents careful manipulation of encrypted data by an attacker to change it's meaning. Specifically, row binding applies strong cryptography to bind all the secure fields of a single row/entity together to mitigate a ciphertext replacement attack.

As an example: Imagine a law enforcement scenario where Suspect-A from Case-A simply copies encrypted field "Secure_SuspectName" from Case-B into Case-A. He overwrites his own name with someone else, without ever having to decrypt anything. Without row binding, this silent attack would effectively accuse Suspect-B and exonerate Suspect-A. Enabling this value increases security since an attack with cherry picked replacements will now raise a security exception when a property/field value is discovered that wasn't part of the object/row at the instant the entire object/row was encrypted or sealed. Binding is performed on writes and verified on reads and adds about 16 bytes to the encrypted data.

Default: true

 

<cipherDbConfig>

All settings related to CipherDB are captured in this XML configuration element. It has several XML attributes as follows:

Name Example Description
migrationPolicy migrationPolicy="ReadOld" The latest version of a given key is always used when encrypting fresh data. migrationPolicy decides what happens when data encrypted by older versions of the encryption key is encountered.

Default: ReadOld

Options are:

NoOld: Entities encrypted with any older key versions will be rejected i.e. NOT be decrypted. Rare but used in cases where older versions of encryption keys were compromised and data encrypted by those keys cannot be trusted anymore. Such data is encountered, it's decrypted as NULLs to protect untrusted data from entering the application.

ReadOld: Most common case where data encrypted with older keys will be decrypted. CipherDB will not queue up any writes to migrate data to the latest data encryption key. This default mode allows you to issue new keys periodically, ensuring no single key every protects too much data.

MigrateOnWrite: Like ReadOld but also allows for live migration of entities from older key versions to the latest key version. The following conditions apply within this mode:

    • The application must issue a SaveChanges() within the same database context (e.g.: same using block) that read the older entity
    • MigrationLimit, a separate configuration parameter, is a positive number
    • The older version of the used key is accessible (i.e. the Security Partition ACLs allow access)

Behind the scenes, entities with older encryption keys are queued up for migration to the latest encryption key upon the SaveChanges(). Since this increases the database write times linearly with number of entities migrated and since it can happen in a live environment you use MigrationLimit to control the impact. Due to access patterns inherent in any application the most frequently used entities are migrated first and migration volume tapers off naturally.

migrationLimit migrationLimit="20" Valid only for MigrationPolicy = MigrateOnWrite.

This specifies how many database entities using the older encryption keys are tracked by CipherDB (upon read) for subsequent key migration related writes.Example: MigrationLimit is set to 20. Within a using block, the application reads 30 entities from the database out of which 25 are still on an older AES encryption key. CipherDB tracks the first 20. The application then decides to write 2 entities to the database, within the same using block as the read. At the instant of the application triggered write, CipherDB will write 2 (Application originated) + 20 (Application read, CipherDB tracked) = 22 entities into the database.

Default: 20

<cipherStorConfig>

All settings related to just CipherStor are captured in this XML configuration element. The XML attributes are as follows.

Name Example Description
reportEventEveryXBytes reportEventEveryXBytes="16000" Determines after how many bytes of processing should CipherStor send an event to the callers to notify it. Default: "0" (never)

Recent blog posts

Migrating existing live data into Crypteron

You’re already live in production. And you have sensitive in the clear. Read this article to see how Crypteron can help.

Encryption, Entity Framework and Projections

Projections in Entity Framework live outside the entity lifecycle. Read more to learn how your can use Crypteron to secure such data.

PCI DSS and key rotations simplified

PCI compliance requires data encryption keys to be changed frequently. Here is how you can do it easily.

Your data-center is not secure and what you can do about it

There is no secure perimeter anymore. Neither in your corporate network nor in your data center. Fight a winning battle armed with self-protecting data rather than a losing one trying to protecting the infrastructure.

Introducing the Crypteron Startup Innovators Program

Qualifying startups get up to 50% off all plans. Tell us how you’re changing the world and the our Startup Innovators Program will support your journey.

6 encryption mistakes that lead to data breaches

If encryption is so unbreakable, why do businesses and governments keep getting hacked? Six common encryption mistakes that lead to data breaches.

Announcing the new Crypteron Community Edition

Starting today you can now sign up for the Crypteron Community Edition for free with no performance limitations.

Data breach response – One click to save your business

Get breathing room – when you need it the most. Respond to a data breach with a single click.

Why We Need Proper Data-At-Rest Encryption: 191M U.S. Voters’ Data Exposed

Adding security at the application level is a large step forward in protecting data from the constant threat of data breaches

How to encrypt large files

CipherStor is blazingly fast! Here we show how to use it within your data-flow pipeline to maintain high performance when encrypting large files.