Helpful Information
 
 
Category: Embedded Programming
Idea proposal: webserver

Purely theoretical, because it'll be months before I finish my other stuff and can actually build this.

I'm thinking of creating a web server that can run on your basic microprocessor - atmel, pic, arm, ti, whatever. Tools aren't even an issue yet.

The idea is this:

One can create a webserver in java, or c++, or whatever other language these days without too much trouble. Hell, you can even create a webserver using some languages that should never, ever, be used for such a purpose. Now, instead of a webserver that interprets code saved as whatever file, one could just have it create pages directly in that language. One would assume that the biggest web companies - like, say, google - don't mess around with PHP or ASP or JSP or any of that garbage, but rather create specialized webservers that optimize the hell out of whatever task needs to be done.

Well, let's extend that idea. We can write a webserver using C, or even assembly. Displaying a static webpage is a matter of sending bytes representing raw HTML, but that's hardly interesting - if I wanted a static website, I'd mail someone a printout of whatever I wanted it to say. So let's make it dynamic - which we can, as long as we can read requests. Really, it's just ...?param1=value1&param2=value2, etc. Now, on your computer, that's a nice socket connection, but here we'll be hardly so lucky - it'll be a stream of bits.

Of course, we also need to make sure that users can access more than one page - index, signup, blah blah. Because of the limitations on how much code can fit on a microprocessor, it actually becomes much smarter for the microprocessor to take a program (webpage) from an external source, compile it, reprogram another partition of itself, and execute that partition, then finally drop the connection. Essentially we're writing a bootloader, except it's a "someone requested a webpage"-loader.

So, on the software side, we gotta create a map of parameters and values, and return some HTML based on that. That's fairly easy.

On the hardware side: take an ethernet wire. Figure out the communications protocol (ie, google). Probably store all incoming and outgoing data in a buffer (flash?) Okay.



So that was the basic part. This has been done. Here's the fun part: databases.

A dynamic website is useless without a database, often. We need to figure out a way to store large amounts of data logically, and read/write it.

On the software side, I'm thinking mysql, only incredibly stripped down. See if I can get mysql working with only the most basic functionality (select, insert, update, delete, and table manipulation; no different databases, no authentication, only a few basic datatypes). Maybe add in optimizing, etc.

On the hardware side, there's only one way I know of to get gigs, or even megs, of cheap flash memory, and that's a flash drive. Use one? Two, mirrored? Three, RAID (one for parity)? Probably one or two. Making an interface for multiple flash drives to be RAID 1 or RAID 5 or whatever would actually be best suited to be a separate task... Also, it's entirely possible to set up a RAID 5 on a single flash drive with three partitions; doesn't prevent against drive death, but it does fix corrupted bits.



Benefits:
Cheapest webserver that's actually useful. If it can hold a few megabytes of data, serve up a webpage in a second or three, and work reliably for 8 or 16 connections, we're talking useful. Slap one of those in an office and run a calendar/scheduling/memo/note-keeping website on it and you've just spent around $75 instead of $750. And it draws almost no power (we're talking batteries). And it's incredibly portable. And we can interface with a zigbee instead of an ethernet if we so choose.

Also, virus-proof? Well, essentially. Your issue might be someone who reads the code and does something like gives it a buffer overflow. Your issue will not be any virus written for any operating system because this has none.


Bad things:
Yeah, you're got possibly the slowest webserver, ever. And one that can only handle a few connections (if more than one, that is, which isn't a given.) And one that's going to be pretty interesting to make webpages for. And one that has a laughably-underperforming database.

Also, it'll probably be capped at something like 1 mbit of data transfer. No idea about that yet.


Counterpoints:
In a small environment, you don't care that it can only handle a few simultaneous connections.

Because it's on the intranet (hopefully!) it's not _terribly_ slow, just, slow. Also, this will probably run faster than an old 286 someone salvaged - no bloat, no slowdown, no decade of registry errors... And only 128 KB/sec is not at all bad for transferring webpages (one would assume no more than, say, 8 KB of html for a small website is very reasonable.) On the other hand, 128 KB/sec is incredibly easy to DOS.

Lastly, how many light applications use anything other than the four basic commands anyways? Hell, you got people in the PHP forum who've been posting for two years and still don't know about unions or joins.





Well, if anyone was interested by my rambling and has thoughts of their own to add, let's have em now.

For a while, I was using a single board computer with an ARM and a fair amount of Debian on it. It came with apache! Pretty amazing. Worked well.

Of course, it didn't have any disk, so it could not deliver a lot of full screen video :-)

So if you pick an ARM based system, it might all be done for you.

Starting from scratch, I'd probably pick Python.

FYI: boards are under $100 and ready to go from
http://www.embeddedarm.com/

Ya, such tools exist, but I'm much more interested in recreating the wheel, as it were - as a student, this is a learning project, not a real tool one would sell.

Anyways, the idea is that it can use cheap, commercially available flash drives to hold quite a bit of space. Yeah, I suppose one could even stream movies off it...

No problem.

A simple HTTP server can be very small. Esp if all you want to do is deliver text files as text/html and text/plain

No need for a bunch of administration controls, SSL support, etc.

its pretty much listen on port 80, verify the parameters and deliver.

To get "good" performance, you would want multi-threading, but you can probably skip that too.

What do you think of the database idea?

A full blown SQL database in an embedded system breaks my brain. Sqllite would be a better choice than MySql, but its still fairly big.

I don't see much of a use case where you would need multiple tables. I can see a very lightweight system that lets you select on one table using a few variables, perhaps with even boolean operations.

Implementing a relational database with basic select, project, join, set union, set difference, and rename. is a one term CS grad student course project. Doing less might take one person about three months.

Hmm. Yeah, it might be a bit too big.

I won't be creating the db from scratch, but rather using existing open source ones for a model... and cutting out almost anything. Still, support for multiple tables.

With the size of modern flash drives and something like 128KB/s, it'd be possible to stream music and use it as a media server with a login system, etc.

I'll consider this more and post back as I think of new things or if I decide to do it.










privacy (GDPR)