Storing data

[RobbyH]

I have a question.
I am using AMS to make an electronic diary.
I am trying to figure out what the best way to store data would be.
AMS seems to deal primarily with ini and text files. I am pretty sure there is a character limit on these types of files. When a user enters whatever they want to record about that day, I want it to be stored in a file on the users system. If I use either the text or the ini file, you could be out of storage space(within that file) very quickly. Any suggestions on how to do this?
I need to be able to access the data, by entering the date name as well.

[Corey]

Text files have no length limit that I know of. /ubbthreads/images/icons/smile.gif Or if there is one it's certainly well beyond your needs for this project unless your "day at a glance" looks like War and Peace.

Corey Milner
Creative Director, Indigo Rose Software (http://www.indigorose.com)

[RobbyH]

heh
Ok, I don't know that there is a limit for a fact.
I just remember copying and pasting a huge amount of text once, and I was alerted that it was to big for notepad, and I had to save it in wordpad. Also, I was thinking about adding everything into one ini file, and retrieving the value when needed. I suppose I could work something out with one or more files...

[Corey]

Once you have your project planned out it will become apparent what limits you need to set for which portions.

For example what is going to be the limit for each daily entry, all day timers have limits, so you need to set up some arbitrary procedures. Once you have that done the rest is a breeze. If you are archiving past days you'll need to set a limit on how many get stored, etc...

Corey Milner
Creative Director, Indigo Rose Software (http://www.indigorose.com)

[TJ_Tigger]

What about using a Global List to store entries for a diary? This might make it more secure rather than using a text file that anyone may open.

[kpsmith]

The file limit message you got is stating that notepad can't handle that much text not that the .txt file format can't handle that much text. I've never heard of size limit for .txt files.

Some other ideas you could consider is to store the data in html or XML since it is essentially a text file with specific formating or tagging.

You would simply create a text file and save it with a .HTML or .XML extension. Either one of these could aid in data indexing and retrieval.

[Corey]

Global lists are no good because they are non-persistent, when you close the app all your data is toast.

XML or HTML aren't advisable because in this context they create a few problems without offering any functionality beyond that of flat text files.

Good ideas though guys...

Corey Milner
Creative Director, Indigo Rose Software (http://www.indigorose.com)

[RobbyH]

Allright, a few ideas. But I was figuring that I would have the program get the date in dd:mm:yyyy format, and that way I could use variables to save without creating anything up front. Even if I could do this, I don't know how I would have AMS create new files. I need some ideas.

[Corey]

Not sure what you mean, but anyhow it's easy to create text files using AMS using a TEX FILE > WRITE action, just check out the help files. I also reccomend to take 5 minutes to browse through all the action names, it will increase your awareness of what's possible and how to do things. They're all pretty much self explanatory at a glance for the most part, i.e. TEXT FILE > WRITE is pretty obvious, etc... Good luck!

Corey Milner
Creative Director, Indigo Rose Software (http://www.indigorose.com)

[Corey]

You also might want to post to our project help forum since this one seems like it might take a few questions to resolve. /ubbthreads/images/icons/smile.gif Forum is here:

http://www.indigorose.com/ubbthreads/postlist.php?Cat=&Board=_project_help

Corey Milner
Creative Director, Indigo Rose Software (http://www.indigorose.com)

[kpsmith]

Not knowing what you are planning to do with the Journal entries when your done or how far you want to take the project.....

I was thinking XML or HTML was a good idea. It would probably be what I'd do if I was building a project for a client.

I would probably provide a text entry field (or 2 or 3) to capture date, title, and text. Then to retrieve the entry at a later date you might want some nice formatting for printing, viewing, or repurposing. I thought HTML would be nice for this since a text file has NO formatting.

You could do the same sort of thing by pulling different parts of a text file into different Text objects on you page but that seems like the hard and less flexible way to do it.

Just depends on how sophisticated you want to get.

[Lorne]

Just a quick note: there is no limit to the size a text file can be -- except of course for any limits imposed by the file system on how large files in general can be...for example, you can't have a single file larger than 2 GB in some versions of Windows. And of course you need to have enough free space on your hard drive for the file...if you only have 5 MB of free space, then 5 MB is your practical maximum size for a text file. /ubbthreads/images/icons/wink.gif

Notepad, however, does have a limit, of 64 KB. But that's just because Notepad sucks. /ubbthreads/images/icons/smile.gif

[Corey]

KP, you don't want to include formatting with raw data. Just the facts maam as they say.

That way you can control/edit your formatting in just one place, the very end source. This principle applies to XML, PHP, Databases, etc...

Corey Milner
Creative Director, Indigo Rose Software (http://www.indigorose.com)

[kpsmith]

Corey, you are definately right when working with databases you should Strip the formatting and keep the data.

My approach was mainly because AMS4 doesn't really handle databases well without some supporting software.

I thought it would be so nice and easy to simply call a .html version of a journal entry to view it in a browser object.

Now that you got me thinking though, I would probably use a some kind of delimited text file and "database" it as you said.

Each line in the text file would be an individual journal entry. I would they have delimiters in that line to break out the date, title, and body of the entry.

You then would just call the the appropriate line in the text file and populate the appropriate fields using get delimited string actions.

The user could then view and edit it and on exit of that entry you'd save any changes.

[RobbyH]

Well, I came up with my own non-secure way of doing it.
I decided simply to incorporate the date into the whole thing, as it is a journal. The whole thing creates folders & files based on %username%. Example %SrcDir%\files\%username\%username%options.
It creates a file for each month, using the %month%%year%.ini for a filename. Not at all secure, but I'm just experimenting anyway. Would there be any way I could have it so that autoplay would encrypt all the files in a users folder after they logout, and decrypt(is that a word?) them when they login?

[Corey]

Sure although it's likely not worth it you could come up with an encryption scheme and apply it or apply someone elses.

A very simple example would be to insert redundant characters or scramble a string.

Corey Milner
Creative Director, Indigo Rose Software (http://www.indigorose.com)

[kpsmith]

You may be able to locate a freely distributable .zip utility that can create or extract a password protected .zip file.

AMS can only extract a zip file I believe.

You could then have each users journals in a .zip file. Extract it to a temporary location when adding, editing, or viewing.
Then zip it back up and password protect it when the user is done.

Not 100% secure as there are ways around it but it will stop the casual techie.

You could also take a more simple approach and just store the data in a non-intuitive location and give the .txt file no extension or a bogus .dat extension. This will stop a lot of non-computer savy users.

I also saw a command line encrypter decrypter once. I'll see if I can dig up the reference. It wasn't freely distributable and required an installation so I didn't pursue it.

[Lorne]

Sounds like a good DLL project for Worm. /ubbthreads/images/icons/smile.gif

[RobbyH]

This is mainly just for learning and fun, but if I was to distribute it, I don't think too many people, except for maybe siblings care to read anothers journal. I liked the sounds of both of your ideas kpsmith, although like you said, AMS only handles extraction of zip files....so no go on that one I guess.
As for the other idea, I can easily rename the files, but how does autoplay handle that? If I tell it a file is an ini file, it will believe me?

[kpsmith]

well, to follow up on your remarks and questions. I did find a command line zip utility that can zip unzip and password files. It's free and freely distributable as well.

http://www.info-zip.org

It's pretty painful learning the correct syntax to make it run so I wouldn't bother if it just for fun.

As far as the file extension, AMS is pretty dumb. If you say a .dat is a .txt or a .go file is an .exe file it will believe you.

So I would think an .ini would be no different.

Just a note to any Indigorose people... This is a good thing!!! Not a complaint. In fact Windows based machines are generally pretty dumb about what file format a file really is so it follows suit in my opinion.

[Lorne]

It has nothing to do with being dumb, or with AutoPlay. /ubbthreads/images/icons/smile.gif That's just the way Windows works.

File extensions aren't magical. In fact, technically they're not even necessary. For example, on the good old Amiga, they were just an arbitrary part of the filename...a file had an extension only if you wanted it to, to make it easier for you to recognize the file type. The extension could be .txt, or it could be .textfileImadewithCygnusEd if you wanted. Or there could be no extension at all. The operating system examined the first couple lines of the file to determine what kind of file it was. This predated Windows by several years (the first Amiga was launched in 1985).

Even in Windows, the file extension doesn't determine the file type. It just determines what program the operating system will use by default to view the file.

But you don't have to let Windows make this decision...you can just load a file directly into a program, and the program will see if it can do anything with the file's contents, regardless of the file extension.

Similarly, if you tell Explorer to run an executable, and you pass it a file called "hihi.dat" or "joe" or "foo.bar" or whatever, it will do what you told it to do: try running that file. If it's actually an executable file (it has the appropriate file header, resource segments, etc.), it gets executed.

[kpsmith]

Thanks for the history lesson Lorne /ubbthreads/images/icons/smile.gif

I guess I just want to make sure IR never programs actions to only except file of appropriate file extensions. That would ruin all my fun with changing filenames to protect (or confuse) the innocent.

For example you could have programmed AMS to only except files for the ini actions that end in .ini, .txt files for text actions, .exe, .bat, .com files for all execute program actions, etc.

I don't know if the flexibility to use files with incorrect file extensions was intentional or just fortunate for us users....But I like it!!!

[_acecombat_]

Post deleted by Brett

[Worm]

Easy there... where did that come from?

[RobbyH]

Really.
Must be that time of month.
I am going to all the trouble because I wanted to try to learn how to use AMS, and that was the only thing I could think of.

[Derek]

hmm! ... Can I really do this?

Lorne, you say:
" If it's actually an executable file (it has the appropriate file header, resource segments, etc.), it gets executed."

How do I get this to happen?

I have removed the exe extension from a file, but upon trying to open it, I am given the "Select the program to open it" dialog.
What should I be doing? I dont know how I go about telling Windows it's really an exe file without putting the ext. back in /ubbthreads/images/icons/frown.gif

[Worm]

Make sure you are using File Execute. If you use File Open, Windows tries to guess what to use to open it by the extension. If you use Execute, you are in control.

[Derek]

Sorry, should have made it clearer.
I was looking to do this from outside AMS and just running it within Win Explorer ... dont know if this is poss without the extension.

[Lorne]

I suspect that it is possible, but I don't know how, exactly. /ubbthreads/images/icons/smile.gif Probably need to go through rundll32.exe.

Interestingly, I can get something to happen by calling an extension-less file using START from the command line in 2k, but that's because there's an entry under Registry\HKEY_CLASSES_ROOT\* on my system. (It loads the file in Omnipad, a free text editor coded by a guy in the forums at arstechnica.com.)

[Derek]

Thanks, Lorne. /ubbthreads/images/icons/smile.gif