.csv-to-database program for real-time updating?

Hey guys,

Been browsing the internet for a month now looking for a solution or a start on this problem I'm going to have over the summer:

I currently have a weather monitoring station hooked up to a datalogger that records wind speeds, rainfall, smoke detection, solar strength, and a whole bunch of other data. It's then saved as a .dat which I can convert into a .csv to open up in Excel. Here's where the networking fun comes in (and where I really start to show my ignorance).

How can I get a .csv file off my computer and onto a database server so I don't have to manually input every drop of rainfall that happens over the course of the day? I was thinking about a .csv to SQL conversion but I don't know any program that can, in real-time, take .csv files, convert them for database input, and spit them into the database.

Feel free to clue me in on what good programs there are out there for managing this type of dillema, and any advice that you have that might help shrink the 100MB of .csv files that I have growing on my local hdd

Didn't really know where else to ask this but you guys (and gals) have always had the best ideas. I can post the modem model and other connection info if need be.

 

Hey I just read your problem I don't know if I understand anything about anything but this is just a suggestion I'm not sure if it will work but I thought I'd try and help.

I think if you can write some fancy SQL (some really helpful site I remember called codewalkers or something has really friendly clued up members and good place to start maybe) you can manipulate the data pretty much anywhich way you want to...


EDIT:

I'm thinking you need some function to check/read the .csv file with some pHp / MySQL wizardary i think you can do this and chuck (write) the data into a MySQL database on a server any way you want.

EDIT 2:

Let me check out some php MySql pre-written codes on hotscripts or something and see if someone has done something like this...2 min

 

Simple csv to MySQL tool:

http://www.hotscripts.com/Detailed/79747.html

Also you might want to check out the other tutorials and stuff on hotscripts here:

http://www.hotscripts.com/search?q=csv+into+php&cat=All

I hope this helps.

 

Thanks a ton for pointing me in the right direction on this one. Those links look like the kind of stuff I need to dive head first into.

 

Hey no problem happy to help! Yeah basically how I would proceed with this is play around with the a copy of your csv file(s) and that "Simple csv to MySQL tool" thing see the results you get in the MySQL database , if they are what you want your sorted!

However, you will probably need to tweak or at least add to the script(s) to get it checking the .csv files automatically I'm thinking, that is where those legends on sites like codewalkers (I hope it still exists) can help you out.

EDIT:

Oh and remember in this type of coding I've found (for what my thoughts are worth) bespoke solutions are usually the best.

 

That's exactly the kind of thing I'm looking into now and throughout the rest of the week

I'm glad there are guys like you out there that share their knowledge freely. I really do appreciate it. Moving from PHP/mySQL to cron jobs to wireless data transfer and file conversion can make your head spin.

 

You're most welcome. I'm not clever. I really struggled with stuff like this and no one helped me out at uni and they all knew how to do it and I kept asking lecturers, 'friends' and classmates for help but they couldn't give a stuff about my issues, when they just had to look at it for like a minute or two and they'd be able to sort my code out - I had the ideas and stuff I just didn't know how to go about them. I read books and tutorials and I still didn't understand what was going on and how it can help me a lot of the time. Coding is very unforgiving scripts sometimes don't work at all sometimes for very small reasons that can be spotted by a fresh pair of eyes really quickly.

Then I found http://forums.codewalkers.com/ and the members there really helped me out with my problems and I'm so grateful for that...the least I can do is share this information with you.

It really gets me when people don't help people out when they know they can easily, we are all gonna die someday so whats the point in not sharing the knowledge? I also find it sad that some people enjoy ripping people off - i.e. they'll help you if you pay them silly money...otherwise if you don't pay they'd rather not help you and even if they do help you they STILL insist on taking their information to the grave (i.e. they solve your problem but don't tell you how they did it)...so so sad...I think it's because these type of morons know that they can take money away from people, but you can't take knowledge away from people...if only everyone just shared their knowledge...we'd all be better off.