question about text encoding, reading from file with MBS encryption

Hey all, I am trying to read in an encrypted text file to a string, decrypt and then create 3 arrays out of the csv data in 3 rows. I am using the MBS Encryption module (functions straight from the example to decrypt/encrypt strings). I am able to decrypt/encrypt the files and strings separately but not a string read from file … The decrypt fails on the first step of the Decrypt function below (DecodeBase64) because it returns an empty string. . The string/file is UTF8. I don’t really get how all these different encodings interact … How should I be getting base64encoded text from a utf (or any) encoding?

put together a small reproduction. In it, I am trying to get the value of tStringD to be the decrypted string contained in the textE file that is created. I use that method to decrypt the file and the string decryption part is used in the app with string manipulation after.

Decrypt Function beginning

Function Decrypt (key, text) 

dim Input as String = DecodeBase64(text)

if lenb(input) < 8 then
  // no salt?
  Return ""  
end if
..
..
End Function

Decrypt String from File

Dim tStringE As String 'encrypted string
Dim tStringD As String 'decrypted string

  Dim tis As TextInputStream
  Dim f As FolderItem
  
    f = GetFolderItem("").Parent.Child("testE.txt")
  
  If f <> Nil Then 
    
    tis = TextInputStream.Open(f)
    
    tStringE = tis.ReadAll
    
    tStringD = Decrypt(App.Num, tStringE) 
    
    break
    
  End

Encrypting/Decrypting File

[code]
Dim fSource As FolderItem
Dim fEncrypted As FolderItem
Dim fDecrypted As FolderItem

fsource = GetFolderItem("").Parent.Child(“test.txt”)
fEncrypted = GetFolderItem("").Parent.Child(“testE.txt”)
fDecrypted = GetFolderItem("").Parent.Child(“TestD.txt”)

If fSource.Exists = False Then
MsgBox “oops”
Else

if EncryptFile(App.Num, fSource, fEncrypted) then
if Decryptfile(App.Num, fEncrypted, fDecrypted) then
MsgBox “OK”
else
MsgBox “Failed to decrypt.”
end if
else
MsgBox “Failed to encrypt.”
end if

End[/code]

Full Decrypt Function

Function Decrypt (key, text) 
// AES 256 plus good key generation
// returns Base64, so you can store in text field in database

dim Input as String = DecodeBase64(text)

if lenb(input) < 8 then
  // no salt?
  Return ""
  
  break
  
end if

key  = ConvertEncoding(key,  encodings.UTF8)


dim iKey as MemoryBlock = key
dim salt as MemoryBlock = leftb(input, 8)

const RoundNumbers = 1000

dim CKey as MemoryBlock
dim CIV as MemoryBlock


if CipherMBS.BytesToKey(CipherMBS.aes_256_cfb128, DigestMBS.SHA512, salt, key, RoundNumbers, CKey, CIV) then
  
  // last 64 bytes are hash
  dim CheckHash as string = input.RightB(64)
  
  Input = Input.midb(9, lenb(input) - 64 - 8)
  
  dim c as CipherMBS = CipherMBS.aes_256_cfb128
  call c.DecryptInit Ckey, CIV
  
  dim output as string = c.ProcessString(Input) + c.FinalizeAsString
  dim outputHash as string = SHA512MBS.Hash(output)
  
  break 
  
  'if StrCompBytesMBS(outputHash, CheckHash) = 0 then
  // ok, bytes are same in hash
  if encodings.UTF8.IsValidData(output) then
    // ok
    dim content as string = DefineEncoding(output, encodings.UTF8)
    
    break
    
    Return content
  else
    // text encoding error? Maybe wrong key?
    break
    
  end if
  break
  'else
  '// hash doesn't work, so wrong key!
  'break
  'end if
  
else
  break
  // failed to make key
  
end if

End function

Base64 uses a narrow range of ASCII characters so reading the data as UTF8 should be fine.

Have you checked that the data in the file is base64 encoded?

[quote=406715:@Kevin Gale]Base64 uses a narrow range of ASCII characters so reading the data as UTF8 should be fine.

Have you checked that the data in the file is base64 encoded?[/quote]

Of course I didn’t, but just changed and still empty.

put together a small reproduction

So I click on the Download button for test.zip, the page seems to reload and … nothing else happens. No sign of the file.

That’s weird, try this one https://share.stonesecurityengineering.com/_NcsRRqBQggJoFR

Thass better.

Ah drat it, of course I don’t have the MBS Encryption module.

But here’s a question. Which is the file that’s supposed to be Base64 encoded? Not test.txt I hope.

[quote=406733:@Tim Streater]Ah drat it, of course I don’t have the MBS Encryption module.

But here’s a question. Which is the file that’s supposed to be Base64 encoded? Not test.txt I hope.[/quote]

fixed, included the base64 converted file (using notepad++ after decrypting file with included method). Note the op has some wrong info now. I am trying to take a file, idc about the encoding I just want to read in the contents of an encrypted file to string then decrypt. How can I handle these encodings, I tried many variations like not using base64 encoding in the first line of the Decrypt function, using the utf8 string right from the file and in the Decrypt function first EncodeBase64 to a new string var and then Decode from there (still empty return value) …

https://share.stonesecurityengineering.com/_NcsRRqBQggJoFR

Any test file you create is likely to be just ASCII (and therefore is UTF-8). Are you sure that by running it through NotePad, you’re not prepending the file with a BOM?

Yea, they’re output in utf8 and notepad++ has a selection for encoding, all files are utf8 without BOM

Then in decrypt you do this:

[code]dim Input as String = DecodeBase64(text)

if lenb(input) < 8 then
…[/code]

and if in the debugger you break on the “if” statement, does the variable “text” contain the base64 from the file? And is “input” empty at that point? (NB is “text” an OK name for a string variable? Asking for a friend.)

[quote=406741:@Tim Streater]Then in decrypt you do this:

[code]dim Input as String = DecodeBase64(text)

if lenb(input) < 8 then
…[/code]

and if in the debugger you break on the “if” statement, does the variable “text” contain the base64 from the file? And is “input” empty at that point? (NB is “text” an OK name for a string variable? Asking for a friend.)[/quote]

text does contain the base64, but the Input var and textE look different

Here’s an alternate question. Notice the Decrypt function says it returns base64. Why does it need to be base64? I dont care about the encoding and I don’t want to deal with that, so how is it returning base64 when the only thing I see regarding that is decoding right at the beginning? How can i make this not return base64 but just a damn string?

This code is confusing.

The EncryptFile function uses CFB128 with AES class.

But Decrypt function uses different cipher!

This can’t work.

Please use same way to encrypt as you decrypt.

No, you read base64 off disk and then decode that to binary, right at the start of decrypt(). That set of binary bytes, however, is only 26 bytes long. But later your code seems to wish to take the leftmost 8 binary bytes as a salt and the rightmost 64 binary bytes as a hash, but the binary is far too short for that.

Then you do a number of steps (no idea what these are for, as I don’t have the MBS Encryption module). I assume this is supposed to produce a text string (as you then check that it is valid utf-8).

Got it, made a copy of DecryptFile to DecryptFromFile and just took the string instead of writing to new file …

[code]Function DecryptFromFile(key As String, sourceFile As FolderItem) As Boolean
// decrypt file with AES and CFB

dim a as AESMBS = NewAES(key)
if a = nil then Return false

dim bi as BinaryStream = sourcefile.OpenAsBinaryFile(false)
if bi = nil then Return false

'dim bo as BinaryStream = destfile.CreateBinaryFile("")
'if bo = nil then Return false

dim IVectorOffset as integer = 0
dim IVector as new MemoryBlock(16)

dim data as string = bi.Read(1000000)

Dim DataString As STring

while lenb(data)>0

dim idata as MemoryBlock = data
a.DecryptCFB128(idata, lenb(data), IVectorOffset, IVector, nil, 0, 0)

'bo.Write idata

dataString = idata.StringValue(0, lenb(data) )

data = bi.Read(1000000)
break

wend

'bo.Close
bi.Close

break

Return true[/code]