Unicode Action Plugin

[boku]

An extremely useful and code cutting solution. Much appreciated Peter!

[Intrigued]

Thanks Ulrich!

I'm still around, just in a read mode with AMS of late. I still try to upload all professionally done, polish free plugins, assemblies, etc. So, if someone gets stuck, needs something. Please post it on the forums and one of us archivers will surely try to accomodate.

[Clue]

What's the common cause for a 12031 error?

Ulrich,

Thanks for the suggestion in the other thread about displaying Japanese Character.

Anyways, I searched and found your plugin and it seemed to be my savior. :yes But I kept on getting this 12031 error when I attempted to use it. I double checked the unicode text file and made sure that it is saved in Unicode format. Well, it actually is just a text file with a line of Japanese char created with Notepad. So what could possibly be the cause of this error?

Thanks for your help.

[Ulrich]

Hello,

could please zip the file and attach it here? The error is expected if the encoding specified does not correspond to the actual source file, or maybe if the source file cannot be translated to the target encoding... I'll have to see what is happening.

Ulrich

[Clue]

Zipped File Attached

Ulrich,

Here you go.

Thanks a lot.

[AMSWaves]

Hi All,
this is possible to call Unicode Function or pass a string as a PWChar with memory plugin an some api programming.
i write two function (Ansi2Uni & Uni2Ansi) with these you can call unicode functions (Uses Memory Plugin 1.0.2.0)

function Ansi2Uni(st)
  blen = (String.Length(st)*2)+2
  wbuf = Memory.Allocate(blen)
  _st = Memory.Allocate(String.Length(st))
  Memory.PutString(_st, st, -1, "UTF8")
   
  DLL.CallFunction("kernel32.dll", "MultiByteToWideChar", "0, 0, ".._st..", -1, "..wbuf..", "..blen, DLL_RETURN_TYPE_INTEGER, DLL_CALL_STDCALL)
  Memory.Free(_st)
  return wbuf
end

function Uni2Ansi(Uni)
  size = DLL.CallFunction("kernel32.dll", "WideCharToMultiByte", "0, 0, "..Uni..", -1, 0, 0, 0, 0", DLL_RETURN_TYPE_LONG, DLL_CALL_STDCALL)
  heap = DLL.CallFunction("kernel32.dll", "GetProcessHeap", "", DLL_RETURN_TYPE_LONG, DLL_CALL_STDCALL)
  ansi = tonumber(DLL.CallFunction("kernel32.dll", "HeapAlloc", heap..", 8, "..size, DLL_RETURN_TYPE_LONG, DLL_CALL_STDCALL))
  DLL.CallFunction("kernel32.dll", "WideCharToMultiByte", "0, 0, "..Uni..", -1, "..ansi..", "..size..", 0, 0", DLL_RETURN_TYPE_LONG, DLL_CALL_STDCALL)
 
  local ret = Memory.GetString(ansi, -1, "UTF8")
  Memory.Free(ansi)
  return ret
end

And see example how you can call MessageBoxW(this is not possible in AMS) :

Text = Ansi2Uni("Hello World")
Caption = Ansi2Uni("Test")

DLL.CallFunction("user32.dll", "MessageBoxW", Application.GetWndHandle()..", "..Text..", "..Caption..", 0", DLL_RETURN_TYPE_INTEGER, DLL_CALL_STDCALL)

so i learn one thing in my Programming Life, that nothing is not impossible in programming.

Enjoy.

[gvanassche]

Ulrich, this really fulfils a need... I'm not sure however it can fulfil my need:

I have text in an RTF file (with Japanese characters) and I want to convert this to a UTF-8 TXT file. Or, I have copied this text from a website, and I want to save the copied japanese UTF-8 text to be save as TXT file in UTF-8.

Would this be possible with your plugin?

(I'm not sure that the clipboard plugin supports UTF-8...)

thanks for your help,

gert

[lasho]

Hello

I'm newbie here and can you help me wiyh this (unicode text to listbox) function?
i have this code in oneclick

I get this code example from here http://www.azman.info/ams/ShareData2.apz

tTextFile = TextFile.ReadToTable(_DesktopFolder.."\\test.txt")

for n=1, Table.Count(tTextFile) do
tLine = DelimitedStringToTable(tTextFile[n], ",")
if Table.Count(tLine) > 0 then
ListBox.AddItem("MyListBox", tLine[1], tLine[2])
else
-- badly formatted line
Debug.Print(tLine[n])
end
end

I try to change with

tTextFile = TextFile.ReadToTable(_DesktopFolder.."\\test.txt") this

to

tTextFile = Unicode.ReadToTable("_DesktopFolder.."\\test.txt", "utf-8");


but appears error "Line 4 Argument 1 must be of type table"


How to fix this error ?

Aslo in global is code

function DelimitedStringToTable(DelimitedString, Delimiter)
tbReturn = {};
local strWorking;
local nPos = nil;
local strData;
local nTableIndex = 1;
local nDelimiterLength = String.Length(Delimiter);

if(nDelimiterLength < 1)then
tbReturn[nTableIndex] = DelimitedString;
return tbReturn;
end

strWorking = DelimitedString;
nPos = String.Find(strWorking,Delimiter);
while(nPos ~= -1)do
strData = String.Left(strWorking,nPos-1);
tbReturn[nTableIndex] = strData;
nTableIndex = nTableIndex + 1;
local nLength = String.Length(strWorking);
strWorking = String.Right(strWorking,nLength - (nPos + (nDelimiterLength-1)));
nPos = String.Find(strWorking,Delimiter);
end
if(strWorking ~= "")then
tbReturn[nTableIndex] = strWorking;
end

return tbReturn;
end

[usbticari]

ASCII UNICODE: TextFile table or string convert issues and Free Solutions

There is an error in the unicode plug-in file. My advice to you "http://www.autohotkey.com/board/topic/9831-uniconv-convert-unicode-cmd/" address the utility program as "uniconv" I suggest you use the program.
Download the program here. uniconv.zip

Example:

.........
File.Run(_SourceFolder.."\\uniconv.exe ", "ucs2 old.txt ASCII new.txt", _SourceFolder, SW_MINIMIZE, true);

tTextFile = TextFile.ReadToTable(_SourceFolder.."\\new.txt");

The features and help files this small utility programs:

Uniconv Help

------------------------------------------------------------------------

Uniconv is a command line utility that uses the Basis Technology C++
Library for Unicode for converting text between encodings and optionally
applying transforms to it.

Help Contents

* Usage
o Examples
* Encodings
o Encodings Quick Reference
* Character Properties
o Properties Quick Reference
* Transforms
o Transforms Quick Reference
* Error Messages
* Copyright Information

------------------------------------------------------------------------

Usage

Uniconv will convert a text file written in a given encoding (click here
for accepted encodings) to another of its accepted encodings. It uses a
command line interface, the usage being as follows:

uniconv [-options] <input-encoding> <input-file> <output-encoding>
<output-file>
[property | transform]*

uniconv
Name of the program to run.

input-encoding required
List the encoding of the input file. Encoding name must be
written in the way listed below.

input-file required
List the name of the file (if in the current directory) or the
path and file name of the file (if not in the current directory)
to be converted.

output-encoding required
List the desired encoding of the ouput file. Encoding name must
be written in the way listed below.

output-file required
List the name of the file to be created in the new encoding (if
in the current directory) or the path and file name of the new
file (if not in the current directory).

property optional
Returns true or false value for characters. A property is
associated with the transform that follows it. Properties not
followed by a transform are ignored. Multiple property-transform
pairs are OK. Multiple properties per transform are also OK. See
Character Properties for more information about how to use
properties, and see below for a quick reference of the
properties available.

transform optional
Changes a property value for designated characters in a file.
Multiple transforms are OK. See Transforms for more information
about how to use transforms, and see below for a quick reference
of the transforms available.

options:
Use these flags at the beginning of the command line, before you
specify the input and output encodings and filenames.
-debug optional
This option will print messages generated by Auto-detect. For
example, if you are converting a Japanese file and the input
encoding is japaneseautodetect, uniconv will list the encodings
it is attempting (sjis, euc-j, etc.) and the results.

-help optional
Displays the copyright information.

-subst optional
Allows you to change the default substitution character. The
substitution character is the character that is used if there is
no direct mapping between characters in a conversion. The
default substitution character is CTRL-Z.

Notes
- All command line arguments are case insensitive.
- Separate properties and transforms with a space.
- If there are multiple properties or transforms, they will be
performed in the order listed.
- The options -debug, -help, -subst, if used, must directly
follow "uniconv".
- * means more than one property or transform is OK.


Examples

to change a file from Shift-JIS encoding to UCS2 encoding
uniconv sjis input.txt ucs2 output.txt

to change a file from ASCII encoding to UCS2 encoding and convert all
uppercase letters to lowercase
uniconv ascii input.txt ucs2 output.txt tolowercase

to keep a file in Shift-JIS and convert all half-width characters to
full-width
uniconv sjis input.txt sjis output.txt ToFullwidth

to keep a file in Shift-JIS and convert all half-width characters to
full-width and all uppercase romaji to lowercase
uniconv sjis input.txt sjis output.txt tofullwidth tolowercase

to keep a file in Shift-JIS and convert only katakana half-width
characters to full-width, leaving romaji half-width characters as-is
uniconv sjis input.txt sjis output.txt katakana tofullwidth



Encodings

Quick Reference: Accepted Encodings
Arabic, ASCII, Big5, BMP, ChineseAutoDetect, cp1251, cp1252, cp437, cp850,
EUC-J, EUC-KR, GB2312, Greek, Hebrew, HZ, ISO-2022-JP, ISO-2022-KR,
ISOLatinCyrillic, JapaneseAutoDetect, JIS_X0201, JIS_X_0208,
KoreanAutoDetect, Latin1, Latin2, Latin3, Latin4, Latin5, Latin6,
Shift-JIS, Thai, UCS2, Unicode11UCS2, Unicode11UTF7, Unicode11UTF8, UTF7,
UTF8



Properties

Quick Reference: Accepted Properties
UppercaseLetter, LowercaseLetter, TitlecaseLetter, ModifierLetter,
OtherLetter, AnyLetter, NonSpacingMark, CombiningMark, DecimalNumber,
OtherNumber, DashPunctuation, OpenPunctuation, ClosePunctuation,
OtherPunctuation, MathSymbol, CurrencySymbol, OtherSymbol, SpaceSeparator,
LineSeparator, ParagraphSeparator, ControlCharacter, OtherCharacter,
UndefinedScript, GeneralScript, Latin, Greek, Cyrillic, Armenian, Hebrew,
Arabic, Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu,
Kannada, Malayalam, Thai, Lao, Tibetan, Georgian, HangulJamo, Hiragana,
Katakana, Kana, Bopomofo, CJKUnifiedIdeographs, Hangul, UndefinedWidth,
Fullwidth, Halfwidth



Transforms

Quick Reference: Accepted Transforms
ToLowercase, ToUppercase, ToFullwidth, ToHalfwidth, ToHiragana,
ToKatakana, Decompose, Compose, ToCombiningMark, ToSpacingMark, Select,
Filter, ToCRLF, ToCR, ToLF, ToParagraphSeparator, ToLineSeparator,
ToCanonical, ToTraditionalChinese, ToSimplifiedChinese, RomajiToHiragana,
RomajiToKatakana, KanaToRomaji, ToLatinNumber, SGMLEntity