Crypto plugin blowfish format

[kolean]

I'm currently trying to encrypt data with blowfish in php script and then decrypt it in AMS with Crypto plugin. Problem is that it just doesn't work :(
I guess the problem is in encryption settings like ecb/cbc and IV. Also decrypting data in PHP which is encoded in AMS doesn't give any meaningful results.
I'm using BlowfishEncryptString/BlowfishDecryptString functions posted in this forum and in PHP I use MCrypt.

Can anyone tell me what IV and settings Crypto plugin uses and also what kind of padding is used.

[Lorne]

I'm currently trying to encrypt data with blowfish in php script and then decrypt it in AMS with Crypto plugin. Problem is that it just doesn't work :(
I guess the problem is in encryption settings like ecb/cbc and IV. Also decrypting data in PHP which is encoded in AMS doesn't give any meaningful results.
I'm using BlowfishEncryptString/BlowfishDecryptString functions posted in this forum and in PHP I use MCrypt.

Can anyone tell me what IV and settings Crypto plugin uses and also what kind of padding is used.

Make sure you are using the latest version of the Crypto plugin. The original version used a proprietary file header which made it incompatible with other blowfish tools.

I'm not familiar with MCrypt; does it handle base64 encoded data? The string functions posted in the forum encode the results in base64 format.

The blowfish class we use internally doesn't specify whether it implements ecb, cbc, or cfb. It also doesn't specify what the initialization vector is. I would have to analyze the implementation in order to figure it out, which I don't have time to do at the moment. It should be possible for you to deduce it by testing Crypto against another tool that allows those settings to be adjusted.

The implementation in the Crypto plugin uses a standard PKCS block padding scheme.

[kolean]

Crypto plugin is the latest version (downloaded today).
MCrypt by itself doesn't handle base64 encoded data, but there are helper functions in PHP to convert to and from base64 so that is not the problem (or then the base64 conversion functions are screwed up).

Crypto plugin seems to behave like blowfish (8 byte blocks, ecb) but generates different results than MCrypt. Initialization vectors seem to only apply to cbc and cfb modes, so Crypto plugin output should be unaffected by that. Perhaps then the problem is the key used for encrypting... Is there any padding or hashing used on the key?

Blowfish looks like somewhat simple to code. It probably could be done in lua :)

[Lorne]

Crypto plugin is the latest version (downloaded today).
MCrypt by itself doesn't handle base64 encoded data, but there are helper functions in PHP to convert to and from base64 so that is not the problem (or then the base64 conversion functions are screwed up).
That might have something to do with it.... Is it possible to write the MCrypt results to a binary file? If so, it might be helpful to compare the results without any base64 encoding.

Crypto plugin seems to behave like blowfish (8 byte blocks, ecb) but generates different results than MCrypt. Initialization vectors seem to only apply to cbc and cfb modes, so Crypto plugin output should be unaffected by that. Perhaps then the problem is the key used for encrypting... Is there any padding or hashing used on the key?

Looking at the code, there is a function that initializes an encryption sieve; it initializes the arrays with values from a couple of random number tables. (Does that constitute an IV?) It also looks like it hashes the sieve with the key before calling the encipher function. I'm not familiar enough with the algorithm to know which one this corresponds to, unfortunately.

Here is the initialization function from the class that we're using:


void CBlowFish::Initialize (BYTE key[], int keybytes)
{
int i, j ;
DWORD data, datal, datar ;
union aword temp ;

// first fill arrays from data tables
for (i = 0 ; i < 18 ; i++)
PArray [i] = bf_P [i] ;

for (i = 0 ; i < 4 ; i++)
{
for (j = 0 ; j < 256 ; j++)
SBoxes [i][j] = bf_S [i][j] ;
}


j = 0 ;
for (i = 0 ; i < NPASS + 2 ; ++i)
{
temp.dword = 0 ;
temp.w.byte0 = key[j];
temp.w.byte1 = key[(j+1) % keybytes] ;
temp.w.byte2 = key[(j+2) % keybytes] ;
temp.w.byte3 = key[(j+3) % keybytes] ;
data = temp.dword ;
PArray [i] ^= data ;
j = (j + 4) % keybytes ;
}

datal = 0 ;
datar = 0 ;

for (i = 0 ; i < NPASS + 2 ; i += 2)
{
Blowfish_encipher (&datal, &datar) ;
PArray [i] = datal ;
PArray [i + 1] = datar ;
}

for (i = 0 ; i < 4 ; ++i)
{
for (j = 0 ; j < 256 ; j += 2)
{
Blowfish_encipher (&datal, &datar) ;
SBoxes [i][j] = datal ;
SBoxes [i][j + 1] = datar ;
}
}
}

When I was testing our implementation of the padding code, I ran into at least one free tool that implemented PKCS padding incorrectly. There is also a bug in the original blowfish source that supposedly isn't fixed in all implementations, but I believe it is fixed in ours. It's possible that MCrypt's results are different for one of these reasons as well.

Blowfish looks like somewhat simple to code. It probably could be done in lua :)

I would bet that it has, somewhere. :)

[kolean]

After some testing with reference C implementation I found out that PHP/MCrypt wants data in big-endian format and on x86 platform data is in little-endian format. Key is processed byte-by-byte and is not affected by that. At least now Crypto-plugin demo (which is based on version 1.0.0.0 btw) produces same results as reference implementation. In PHP I need to do endianness converion on input data and results to make it look same.

I have to do some more tests at work where I have full AMS and Crypto plugin but I believe that was the problem.

[kolean]

That was indeed the case. Now I can encrypt/decrypt in PHP.

Below is sample code I needed to get it working. Feel free to use it as you wish. Code might contain bugs so remember to test it :)


//Pads data to mod 8 bytes, PKCS#5 style
function pad($data) {
$padlen = 8-(strlen($data) % 8);
for ($i=0; $i<$padlen; $i++)
$data .= chr($padlen);
return $data;
}

//Unpads data (and checks if pad is valid)
function unpad($data) {
$padlen = ord(substr($data, strlen($data)-1, 1));
if ($padlen>8)
return $data;

for ($i=strlen($data)-$padlen; $i<strlen($data); $i++) {
if (ord(substr($data, $i, 1)) != $padlen)
return false;
}

return substr($data, 0, strlen($data)-$padlen);
}

//Swaps byte order (little-endian <-> big-endian)
function swap($data) {
$res="";
for ($i=0; $i<strlen($data); $i+=4) {
list(,$val) = unpack('N', substr($data, $i, 4));
$res .= pack('V', $val);
}

return $res;
}

require_once 'Crypt/Blowfish.php';

$key="key";
$data="Hello world";
$bf =new Crypt_Blowfish($key);

$enc = swap($bf->encrypt(swap(pad($data))));
echo base64_encode($enc) . "\n";

$real = unpad(swap($bf->decrypt(swap($enc))));
echo $real;



It works also with MCrypt:

$enc = swap(mcrypt_encrypt(MCRYPT_BLOWFISH, $key, swap(pad($data)), 'ecb'));
echo base64_encode($enc);

$real = unpad(swap(mcrypt_decrypt(MCRYPT_BLOWFISH, $key, swap($enc), 'ecb')));
echo $real;

[kolean]

Encryption/decryption works but writing to file and reading there is absolutely too slow. I need to do 20-50 decryptions per page (and much more on some) and that makes noticeable delay in rendering. Performance is something like 100 decryptions per second.

Crypto plugin with in memory operations is required for this (for both blowfish and base64). Any possibilities to get (quick) update to Crypto plugin to support those operations? Otherwise I have to code my own plugin to handle those.

[Lorne]

Ah, the endianness...I briefly considered that at one point (there are defines in the class header to select between ABCD, DCBA, etc.) but dismissed it because I didn't think it would apply. Interesting that MCrypt wants big-endian byte order...I wonder what kind of hardware your web server is running on? :)

Encryption/decryption works but writing to file and reading there is absolutely too slow. I need to do 20-50 decryptions per page (and much more on some) and that makes noticeable delay in rendering. Performance is something like 100 decryptions per second.

Could you decrypt the data all at once in advance? You could use a progress bar to make it seem to take less time.

Are you sure you need to base64 encode the data, though? That adds to the decoding time, and inflates the data by about 33% on average.

Crypto plugin with in memory operations is required for this (for both blowfish and base64).

Which operations, specifically? Blowfish decrypt from string to file?

Any possibilities to get (quick) update to Crypto plugin to support those operations? Otherwise I have to code my own plugin to handle those.

Can't promise anything on being able to produce a quick update...I'm not sure when the next plugin update pass is scheduled. But I could look into it.

[kolean]

Ah, the endianness...I briefly considered that at one point (there are defines in the class header to select between ABCD, DCBA, etc.) but dismissed it because I didn't think it would apply. Interesting that MCrypt wants big-endian byte order...I wonder what kind of hardware your web server is running on? :)
Just plain simple Intel Celeron laptop (with Windows) and Athlon64 (running Linux) computers. I found out that endianness problem when I tried to encrypt all nulls string with all nulls keys. There I could see that the results were "backwards".

Could you decrypt the data all at once in advance? You could use a progress bar to make it seem to take less time.

Are you sure you need to base64 encode the data, though? That adds to the decoding time, and inflates the data by about 33% on average.
There is about 30MB of data (as in SQLite database size), probably about 15MB of strings. Decrypting all feels a bit wasteful. (Especially now that I put lesser obfuscation to frequently used data and made it perform better).

Now it seems that AMS itself is the slowest point. There is annoying flickering when I update about 40 labels (texts and visibility). Application.SetRedraw(false) helps a little but doesn't cure the problem. There is also a bug in Image.SetPos/SetSize that they fail mysteriously when redraw is turned off. I even got AMS app to crash few times.

Which operations, specifically? Blowfish decrypt from string to file?
Decrypt (and encrypt) from string to string. Useful when you store encrypted data in (SQLite) database and have to put it to a label.