KSP-Modular

I have been working on KSP-Modular and am quite happy with how it is turning out.  I have decided to implement the entire database structure using SQLite.  I am pretty happy so far, and it also has the added benefit of forcing me to finally learn some SQL.

The current size is over 850 lines.  Pretty huge.  It includes a fair amount of the core system too:

• The User Buffer system
• A small updater setup for KSP itself
• The routines for adding handler applications
• Help system for most of implemented routines
• Some spiffy web tools (way more to come)
• A highly rudimentary script parser

So yeah, it's pretty nice so far.  I still want to get some more program manager routines in place right away.  I am also working on a script that will test all of it's routines.  This is for two purposes, one being that it would be handy to show me if I break any existing routines, and second it will be useful for generating data for libzip.  I am hoping to be able to pack the whole system using libzip, and it will definitely require a custom library setup.

Update - Hookless keylog

I now have some methods in place to keep track of what text is going to what window.  This is handy if someone is typing a bunch between multiple windows (IE a browser and a messenger).  I am still working on getting the data logging done, and making some type of high speed time stamp routine.

The current size is now around 20kb, and the goal is not to exceed 25.

Current Project - Hookless keylogger

It's coming along ok, it is a bit slow because of my job and because my wrist is messed up (and it's tough to type with a brace).

Anyhow, it's looking good, and is currently only 17kb.  Super tiny.

I haven't decided whether or not it will be published to the site when done since it *could* be used for, well, less then great purposes.

KSP-Modular and Perl's "use strict"

This is the first large project that I have enabled both warnings and strict on.  Generally I don't just because most stuff will happily run even when it wouldn't under those modes.

This is intended to be used by more then just me eventually though, so I decided to switch them on tonight and....

~140 errors.

Wow.

Having looked through it, most are simply undeclared variables, a common place thing in my programs.  Well, it is now all clear.  Komodo says no errors, and running perl -c KSP-Modular.pl says the syntax is good.  I'll probably send the revised version to the site tomorrow after some more revisions.

KSP-Modular - Updated version on site

The newest version of the code has been uploaded to the site.  It is still considered version 0.0.1 since it has never been usable.  The version won't increment until it has been considered usable.

Latest code snapshot is over 620 lines, and includes the working stages of the user buffer system, and a conversion of the web link extractor to use the working buffer.

KSP-Modular : User buffer system

I wanted to create a way to store data for user use.  The goal was to create memory space which:

• Acted like an array (basically just a list that could be manipulated easily)
• Was able to store session data in memory without doing hard drive storage.
• Had nameable lists, so you didn't have to remember just a computer assigned address

After thinking about it on and off for about 3 days, I believe I have a solution.  My method is to use an array of arrays.  Now, everyone knows that arrays are managed through indexes, which doesn't fit my last goal.  So, my method for that was a name-to-index hash.  When creating a new data set, the system pushes a blank array into the main array, figures out it's index, and maps that in the hash to your specified name.

The really cool thing is that this method appears to work.  I really would not have been surprised if it didn't.  I am still working on some of the methods for it, and then I need to integrate it into the code.  It is currently in it's own program for demo purposes.

I'm pretty psyched that this idea works though.  It should be a great way of doing what I had wanted.

KSP-Modular - Over 500 lines

Yep, I just noticed that it is currently 511 lines long.  There is still a great deal to be done though, I won't be shocked if it crosses 1,500 lines, maybe by a lot.

Just a quick note

The version of KSP-Modular found on the site it NOT the latest version.  As a general rule, I will post on here when new software is uploaded to the site.

KSP-Modular - web.link.extract

The web link extractor is working great.  Here is an example link dump from the Kain Research homepage

---------- Link list start ----------
http:www.kainresearch.com/images_gen/Kain_Research_Banner.jpg
http:www.kainresearch.com/services.html
http://kainresearch.blogspot.com/
http:www.kainresearch.com/services.html
http:www.kainresearch.com/software.html
http:www.kainresearch.com/chemistry.html
http:www.kainresearch.com/research.html
http:www.kainresearch.com/projects.html
http:www.kainresearch.com/service_que.html
http:www.kainresearch.com/personal_pages.html
http:www.kainresearch.com/php/CMD_Journal/Viewer.cgi
http:www.kainresearch.com/poe_index.html
http:www.kainresearch.com/links.html
---------- Link list end ----------

The extractor module I am using extracts just the relative paths present in the html code, so I wrote a subroutine that maps them out fully.

I still have some testing to do in the regex inclusion/exclusion subroutines, but I am very happy with how it is coming along.

KSP-Modular

The web site has been updated to show this project now. 

Additionally, I have redone the better part of it to use locally scoped variables as per the last post.  It is working well.

KSP-Modular variable naming conventions

I have been working (as stated in the last post) on some web tools.  My original intention was to name variables after the subroutine they were in.  This is not going to work.  You see, with the easier sorting of subroutines, you end up getting subroutines with names such as:

handler_web_link_extractor_fix

Which means I was ending up with variables in there named incredibly long names such as:

handler_web_link_extractor_fix_base_url_array

Which is bad enough, but even worse when concatenating 2 or 3 of those on a line.  Then we have stuff way, way, way past the 80th column.  I don't like that, I generally will try as hard as possible to stay within 80 columns.  At the very least I try very hard to stay on the screen.  I use a 1680x1050 wide screen display, so if it isn't fitting on there, I figure some changes need to be made.

After looking around a bit, it looks like I will simply start making gratuitous use of the "my" command to just make stuff local.  I feel this should work way better, and I will start fixing my variables soon.

Shell program

All development on all shell overlays has been paused for full development on KSP-Modular.  KSP-Modular is now the official development console.

If you actually follow this blog, you may be thinking "Wait, wasn't a shell program announced a page back?"  Yes, indeed it was.  KShell was a design to build a shell overlay in C.  The reasoning behind development in C was first and foremost because I could produce a binary.  Second, it was for it's small size and speed.  I have since found libzip for Perl though.  It allows binaries to be constructed fairly minimally.  That takes care of the first two reasons.  It is not of course as fast as C (by a pretty good amount actually).  Not bad though.  It runs fairly well.

So, given the chance to use Perl instead, it is a nicer choice.  It is easy to develop it, and incredibly powerful and flexible.

The "-Modular" part of the name is actually because I had started work on a KSP script, until I noticed that it would end up being very tough to maintain.  The redesign has a much better layout already.

It will include nice ways to extend it as well.  Including built in abilities to install packages of add-on scripts and their documentation.

I have the path manager pretty much ready, and am working on some link extraction tools.

Visual Basic - Experiences

I have been working with Visual Basic (Express Edition, 2008 Beta, Orcas), and I have to say, it's nice.  It doesn't have the same low level gritty feeling of C, but that is alright.  I would still rather do console work in C and Perl (and console work is most of what I do), but for GUI stuff, this is great.

It's nice to be able to drag and drop your entire UI, and then just have to write the commands.  Anyone who has tried, say, Perl/Tk knows how much work goes into just making a very basic UI.  This is quite different.

I'm also happy with the binary size, I was thinking it would be large, but my current work is only ~50 kb.  Not bad.

Anyone looking to break into GUI programming, you may want to take a look at the Visual Studio Express editions.

Oh, and Visual Web Developer Express is quite nice too.

LC2PD - Visual Basic Edition

I have started learning some Visual Basic.  I've meant to for awhile, and have had the proper software (i'm using Express Edition, though I have a copy of VS Standard that I got at a Microsoft Presentation).  As per usual, the first thing I am converting is LC2PD, which is pretty much done.  It doesn't have safe guards in place, such as checking for proper input, but it does work.

I haven't uploaded it yet, because I may try to put in safe guards, but will post when it is up.

Database - Updates - Delete

Delete is now implemented.  I will start commenting and documenting the code when I have time, but it is usable now at least.

It is now up for download, both source and executable in a zip file.  You can find it on the project page here.

Database - Updates - Help and Set

The set routine has been modified to update entries rather then create duplicates now.  Also, some small bugs were fixed that likely would have caused trouble later.

A small help entry has been placed in as well, by using the -h switch alone at startup.

Work has started on the delete method, it is expected to be done today as well.

Current Projects - Database and USB-Menu

The projects currently at the top of the stack are now Database and USB-Menu (in that order).  Database requires heavy modification to support some new features.  These include:

• Key removal
• Key entry without causing duplicate keys
• Better documentation
• Help system

Some other features being tossed around include dynamic compression of databases, MD5 checksum use to rule out corruption, and remote database access via FTP.  The practicality of these features is being considered. 

Once the database is up, work will commence on the USB-Menu, since it relies on the database.

Website updates

I've done a bunch of work on the site yesterday and today.  I have been careful not to create any broken links, but please contact us if you find any.

I would say which areas have been updated, but at this point, I think it may be all of them.  Have a look around.

Not gone, just busy

You'll see if you refer to my personal blog here that I have been very busy working on a script for PoE2.  Being the first CGI I have ever done, it has proven itself to be rather difficult.

This doesn't mean we aren't available for other things, and it doesn't mean other things aren't being worked on.  It just means this is the primary concern pending anything else.

The journal script is coming along well though, and has taught me a fair amount about CGI.  I feel like parts of the site may be translated to CGI in the future.  For instance, I may write a CGI handler for the code repository.

Anyway, just wanted to update on why the site and this blog haven't seen much attention.  See how hard we work on things :-)

New Website

Unless you are coming here directly, you may have noticed our new site.  Kain Research now has hosting for our domain.  You can see our site at http://www.kainresearch.com.  There is some stuff there, but not a ton yet.

Updates will ensue as often as I can manage.  This will stay the official news source though, since it is already here, and quite a nice setup.

Drop by and have a look.  Let me know if you have any suggestions.

KShell - The (new) Kain Research Shell Overlay

Rather then continue to write my shell system in C (which was titled PShell), I decided to write it in C++.  I wasn't too far anyway, and am already making better headway in C++.

I also renamed it to KShell, for Kain Shell.  I had considered this to begin with, but decided not to seeing as how there is an existing shell called Ksh (Korn Shell).  I decided to change it anyway, since I plan on always calling it "KShell", and not shortening it.

More news when I get farther in.

Monocle - Updater

Work on the updater is going forward.  I am implementing it in NewLisp for the time being, since I can do it faster.  I am turning several other parts into NewLisp parts as well.  As soon as the system is usable, I will work on porting them to C parts (since C compiles smaller, and in technicality, NewLisp doesn't compile at all).

Fun stuff.

The logger is the only part that is written in C, and currently functional.  I may in the future port it to NewLisp, but since it is already in the target language, and already works, it's not a priority right now.

PShell - A small setback

Being a couple hundred lines in...I made a mistake.  I haven't written a makefile yet, so I was just manually compiling the code.  Unfortunately, I have been sick lately, and when combined with not sleeping a heck of a lot... Well, I'm sure you've all been there.

So, long story short, I entered a wrong command to GCC somewhere along the line, and managed to have the compiled output OVERWRITE my source file.  When I hadn't yet made a backup.  Yes I know, rookie mistake.

Luckily a goodly amount of the code is in separate source files, with the idea of modularizing it for future use.  So only the main code was destroyed.

This has prompted me to start work on a small script (Perl) to make backing up much easier for me.  It is very close to done.

I am taking a slight break from coding for a couple days though since I am aggravated at having destroyed that code. 

I'll probably be yo-yoing more.

Side Project - Nearly complete

My little script to compile single file programs is practically done.  I am just making small program flow changes.

Started work on a portable shell-overlay.  I built something similar to this idea a number of months back, but I wrote it in Perl.  I want one that is faster, and more portable.  While I do realize that in some respects Perl is more portable then C (i.e. it works out-of-the-box on Windows or Unix), it requires Perl, which most people don't have.

I suspect this could be an ongoing project, and have decided to just trade off development of Monocle and this program (dubbed PShell for now, short for Portable Shell).  I'll just work on whatever I can think of, on whichever I feel like.

Small side project

I have started work on a very small side-project.  I quite often end up spending time away from my computer, but by others that I could use.  I am writing a small C program that can read in my include and library setup for compile commands, and build a batch file to build files.

The idea is to put this on my USB stick, and then when I am out I will have a simple way to compile my programs without having to worry about setting up all the new include paths.

It is a very simple design, and I figure it will be done fairly quickly.

Code written today:

145 lines of C

06102007 - Monocle Changes

I have decided to remove certain functions from Monocle version 1.0 in order to hopefully get a working product sooner.  One such option is the encryption system I had planned.

The plan was to make all data files capable of being encrypted, and the system would pretty much take care of encrypting/decrypting them by itself.  Unfortunately, that is a fair amount of work for me, and really is more of a fun thing then a usable option.  Not saying it won't be there eventually, but not for now.

It doesn't help that the test web-server I am using seems to randomly decide which files I can and can't download.  I can upload everything just fine, but some come back 404 when I try to download them.  I'm really not sure what to think there.  It's a friend's web hosting account he said I could use.  I may just switch over to using Apache, since it is already installed.

Still working on updater for now.  The defaults are there, but there is a long way to go.

Projects

Since I have this new blog, I may want to speak a bit about my current projects.  Thing is, before I got this blog, I already did a bit of a write up on my personal blog.  It can be found here

Personal blog

From now on of course, all data will be here.  There is not a ton else going on, just some side projects when I feel like it.

Welcome to the official Kain Research blog

I decided to separate my Kain Research blog from my personal blog, so here goes.  From now on all Kain Research posts and progress data will be found here.

I am working on Monocle still.  Logger is working, net installer is working.  Not much else is functioning, but the current project is the updater.

Progress will be posted here when it's made.