This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
https://github.com/RickStrahl/WestwindToolkit/tree/master/Westwind.Data.MongoDb
The problem with the .NET driver is that it works with generics and strongly typed objects so that have to be typed to pass in. It might be possible to add some additional code that just works of strings, (ie. the standard MongoDb json query format), but I'm not quite sure just what limitations you would run into with that because a number of things a require special sytnax. Might be worth playing around with though.
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
So I poked around with this a little bit and it looks like this might just work with the .NET library. I managed to create some generic helpers that can run queries, retrieve single entities and save data using raw JSON strings. IOW, I have a .NET component that takes JSON string input for various operations, and then performs the operation and returns back JSON results (for queries).
So as a proof of concept I think it works. The four operations that I worked on are: Find(), FindOne(),FindAll() and Save(). I think that's all that's really needed except for aggregation framework features.
Interesting idea - I had thought about this before but kind of decided that this is kind of a silly idea that involves too much overhead. The .NET piece of it is actually really fast. I think the biggest overhead is the FoxPro JSON deserialization piece (which already uses .NET anyway) and creating large objects. The recent changes in wwJsonSerializer should help with this however.
An interesting side idea might be to create a simple REST API using Web API to provide Web access for this so you could in fact use HTTP to do these same data operations directly over the Web, but under your application control (rather than a Mongo specific REST server).
To actually make all of this work properly though would take more than a few hours though :-) All the testing I did so far is just the .NET piece with methods with simple singnatures that are callable from FoxPro and using my existing Westwind.Data.MongoDb library. To make this work specifically for FoxPro that code would have to be broken out into a specific component that just supports the FoxPro features more generically outside of a business object.
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
I tried this:
DO _config
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Save('{ firstname: "Rick" }',"Customers")
Got this error message.
OLE IDispatch exception code 0 from wwDotNetBridge: Method 'System.DBNull.SaveFromJson' not found...
I have to get back up to speed on Mongo. I'm guessing that "Customers" is a collection in the Test Database?
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
You need to connect first.
I've updated the code on github. If you pull again, you should now get a Test.prg that you can just run without using FoxUnit.
+++ Rick ---
I now have the loMongo Object.
I tried this:
DO _config
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Save('{ firstname: "Rick" }',"Customers")
Got this error message.
OLE IDispatch exception code 0 from wwDotNetBridge: Method 'System.DBNull.SaveFromJson' not found...
I have to get back up to speed on Mongo. I'm guessing that "Customers" is a collection in the Test Database?
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
You need to connect first. Look at the example:
https://github.com/RickStrahl/wwMongoDb#save-data-and-create-dbtable-if-it-doesnt-exist
If you use strings rather than objects or values, you have to pass the llIsJson parameter:
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/FoxMongoTest")
? loMongo.Save('{ firstname: "Rick" }',"Customers",.T.)
? loMongo.cErrorMsg
This will create the database and collection on first run.
I've updated the code on github. If you pull again, you should now get a Test.prg that you can just run without using FoxUnit.
+++ Rick ---
I now have the loMongo Object.
I tried this:
DO _config
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Save('{ firstname: "Rick" }',"Customers")
Got this error message.
OLE IDispatch exception code 0 from wwDotNetBridge: Method 'System.DBNull.SaveFromJson' not found...
I have to get back up to speed on Mongo. I'm guessing that "Customers" is a collection in the Test Database?
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
If you haven't looked at NEO4J or another GRAPH database, they're really interesting. Gartner says that 25% of the Fortune 500 companies will be using it by 2016. It's a whole different way of analyzing data.
Thanks again for your help, und "Guten Rutsch ins Neue Jahr!"
Marty
You need to connect first. Look at the example:
https://github.com/RickStrahl/wwMongoDb#save-data-and-create-dbtable-if-it-doesnt-exist
If you use strings rather than objects or values, you have to pass the llIsJson parameter:
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/FoxMongoTest")
? loMongo.Save('{ firstname: "Rick" }',"Customers",.T.)
? loMongo.cErrorMsg
This will create the database and collection on first run.
I've updated the code on github. If you pull again, you should now get a Test.prg that you can just run without using FoxUnit.
+++ Rick ---
I now have the loMongo Object.
I tried this:
DO _config
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Save('{ firstname: "Rick" }',"Customers")
Got this error message.
OLE IDispatch exception code 0 from wwDotNetBridge: Method 'System.DBNull.SaveFromJson' not found...
I have to get back up to speed on Mongo. I'm guessing that "Customers" is a collection in the Test Database?
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
Yes, I just re-ran a small test that inserts 10,000 records in about 7.5 seconds:
************************************************************************
* RunManyInserts
****************************************
FUNCTION RunManyInserts()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")*** Recommend you assign your own ids for easier querying
ADDPROPERTY(loCustomer,"_id",SYS(2015))
ADDPROPERTY(loCustomer,"FirstName","Rick")
ADDPROPERTY(loCustomer,"LastName","Strahl")
ADDPROPERTY(loCustomer,"Company","West Wind")
ADDPROPERTY(loCustomer,"Entered", DATETIME())loAddress = CREATEOBJECT("EMPTY")
ADDPROPERTY(loAddress,"Street","32 Kaiea")
ADDPROPERTY(loAddress,"City","Paia")
ADDPROPERTY(loCustomer,"Address",loAddress)loOrders = CREATEOBJECT("Collection")
ADDPROPERTY(loCustomer,"Orders",loOrders)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",121.10)
loOrders.Add(loOrder)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",44.22)
loOrders.Add(loOrder)
lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
loCustomer._id = SYS(2015)
loMongo.Save(loCustomer,"Customers")
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages inserted in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
which is pretty good, BUT most of the overhead in that is actually the FoxPro object to JSON conversion (all requests send JSON but the methods accept objects that are serialized to JSON).
Using JSON directly is a bit faster providing 10,000 records in only 3.5 seconds!
************************************************************************
* RunManyInsertsWithJson
****************************************
FUNCTION RunManyInsertsWithJson()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
TEXT TO lcJson TEXTMERGE NOSHOW
{
_id: "<<loMongo.GenerateId()>>",
FirstName: "Rick",
LastName: "Strahl",
Company: "West Wind",
Entered: "< <TTOC(DATETIME(),3)> >",
Address: {
Street: "32 Kaiea",
City: "Paia"
},
Orders: [
{ OrderId: "ar431211", OrderTotal: 125.44, Date: "< <TTOC(DATETIME(),3)> >Z"},
{ OrderId: "fe134341", OrderTotal: 95.12, Date: "< <TTOC(DATETIME(),3)> >Z" }
]
}
ENDTEXTloMongo.Save(lcJson,"Customers",.T.)
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages updated in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
* RunManyInsertsWithJson
Although the latter is faster you have be very careful building JSON strings by hand as values have to be formatted properly. Strings have to be properly encoded and dates have to be new Date() formatted etc. so in general I hihgly recommend using FoxPro objects to ensure the serialization is proper.
+++ Rick ---
Looks great - Added 1000 records in 1.4 seconds. That's going to be good for me!.
If you haven't looked at NEO4J or another GRAPH database, they're really interesting. Gartner says that 25% of the Fortune 500 companies will be using it by 2016. It's a whole different way of analyzing data.
Thanks again for your help, und "Guten Rutsch ins Neue Jahr!"
Marty
You need to connect first. Look at the example:
https://github.com/RickStrahl/wwMongoDb#save-data-and-create-dbtable-if-it-doesnt-exist
If you use strings rather than objects or values, you have to pass the llIsJson parameter:
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/FoxMongoTest")
? loMongo.Save('{ firstname: "Rick" }',"Customers",.T.)
? loMongo.cErrorMsg
This will create the database and collection on first run.
I've updated the code on github. If you pull again, you should now get a Test.prg that you can just run without using FoxUnit.
+++ Rick ---
I now have the loMongo Object.
I tried this:
DO _config
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Save('{ firstname: "Rick" }',"Customers")
Got this error message.
OLE IDispatch exception code 0 from wwDotNetBridge: Method 'System.DBNull.SaveFromJson' not found...
I have to get back up to speed on Mongo. I'm guessing that "Customers" is a collection in the Test Database?
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
what I want to do is:***********************************************************************
FUNCTION Album(loAlbum)
lcVerb = Request.GetHttpVerb()*** Handle alternate verbs with same URL with
*** separate method since we don't have method overloading in VFP
IF (lcVerb == "POST" OR lcVerb == "PUT")If lMongoIsLive
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/MusicStore")
lcJson = request.GetRawFormData()
loMongo.Save(lcJson,"Albums",.T.)
else
RETURN THIS.SaveAlbum(loAlbum)
endi
ENDIF
Yes, I just re-ran a small test that inserts 10,000 records in about 7.5 seconds:
************************************************************************
* RunManyInserts
****************************************
FUNCTION RunManyInserts()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")*** Recommend you assign your own ids for easier querying
ADDPROPERTY(loCustomer,"_id",SYS(2015))
ADDPROPERTY(loCustomer,"FirstName","Rick")
ADDPROPERTY(loCustomer,"LastName","Strahl")
ADDPROPERTY(loCustomer,"Company","West Wind")
ADDPROPERTY(loCustomer,"Entered", DATETIME())loAddress = CREATEOBJECT("EMPTY")
ADDPROPERTY(loAddress,"Street","32 Kaiea")
ADDPROPERTY(loAddress,"City","Paia")
ADDPROPERTY(loCustomer,"Address",loAddress)loOrders = CREATEOBJECT("Collection")
ADDPROPERTY(loCustomer,"Orders",loOrders)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",121.10)
loOrders.Add(loOrder)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",44.22)
loOrders.Add(loOrder)
lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
loCustomer._id = SYS(2015)
loMongo.Save(loCustomer,"Customers")
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages inserted in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
which is pretty good, BUT most of the overhead in that is actually the FoxPro object to JSON conversion (all requests send JSON but the methods accept objects that are serialized to JSON).
Using JSON directly is a bit faster providing 10,000 records in only 3.5 seconds!
************************************************************************
* RunManyInsertsWithJson
****************************************
FUNCTION RunManyInsertsWithJson()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
TEXT TO lcJson TEXTMERGE NOSHOW
{
_id: "" ,
FirstName: "Rick",
LastName: "Strahl",
Company: "West Wind",
Entered: "< <TTOC(DATETIME(),3)> >",
Address: {
Street: "32 Kaiea",
City: "Paia"
},
Orders: [
{ OrderId: "ar431211", OrderTotal: 125.44, Date: "< <TTOC(DATETIME(),3)> >Z"},
{ OrderId: "fe134341", OrderTotal: 95.12, Date: "< <TTOC(DATETIME(),3)> >Z" }
]
}
ENDTEXTloMongo.Save(lcJson,"Customers",.T.)
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages updated in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
* RunManyInsertsWithJson
Although the latter is faster you have be very careful building JSON strings by hand as values have to be formatted properly. Strings have to be properly encoded and dates have to be new Date() formatted etc. so in general I hihgly recommend using FoxPro objects to ensure the serialization is proper.
+++ Rick ---
Looks great - Added 1000 records in 1.4 seconds. That's going to be good for me!.
If you haven't looked at NEO4J or another GRAPH database, they're really interesting. Gartner says that 25% of the Fortune 500 companies will be using it by 2016. It's a whole different way of analyzing data.
Thanks again for your help, und "Guten Rutsch ins Neue Jahr!"
Marty
You need to connect first. Look at the example:
https://github.com/RickStrahl/wwMongoDb#save-data-and-create-dbtable-if-it-doesnt-exist
If you use strings rather than objects or values, you have to pass the llIsJson parameter:
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/FoxMongoTest")
? loMongo.Save('{ firstname: "Rick" }',"Customers",.T.)
? loMongo.cErrorMsg
This will create the database and collection on first run.
I've updated the code on github. If you pull again, you should now get a Test.prg that you can just run without using FoxUnit.
+++ Rick ---
I now have the loMongo Object.
I tried this:
DO _config
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Save('{ firstname: "Rick" }',"Customers")
Got this error message.
OLE IDispatch exception code 0 from wwDotNetBridge: Method 'System.DBNull.SaveFromJson' not found...
I have to get back up to speed on Mongo. I'm guessing that "Customers" is a collection in the Test Database?
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
I wouldn't recommend doing this.
Capturing the raw JSON is dangerous - anybody could send you whatever JSON they wanted and get that pushed straight into the database.
Here's what you should do:
- Capture the loAlbum object
- Validate it and make sure it has valid data in it
- Then save the object
There are more reasons for this than security as well. MongoDb doesn't work with 'regular' JSON - it uses a custom JSON dialect which requires special syntax for at least dates - stock JSON uses ISO formatted date strings, Mongo requires that dates uses either new Date() or ISODate syntax. The object conversion inside of the wwMongo object handles that for you but can't do that for straight JSON you pass as a string.
So - I highly recommend you use objects for saving - always, unless you have a specific reason not to do that.
+++ Rick ---
Now comes the exciting part - Modify the Music Store to use Mongo. I have to do some more checking into wwDotNetBridge, but I think what I need to do is:
what I want to do is:***********************************************************************
FUNCTION Album(loAlbum)
lcVerb = Request.GetHttpVerb()*** Handle alternate verbs with same URL with
*** separate method since we don't have method overloading in VFP
IF (lcVerb == "POST" OR lcVerb == "PUT")If lMongoIsLive
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/MusicStore")
lcJson = request.GetRawFormData()
loMongo.Save(lcJson,"Albums",.T.)
else
RETURN THIS.SaveAlbum(loAlbum)
endi
ENDIF
Yes, I just re-ran a small test that inserts 10,000 records in about 7.5 seconds:
************************************************************************
* RunManyInserts
****************************************
FUNCTION RunManyInserts()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")*** Recommend you assign your own ids for easier querying
ADDPROPERTY(loCustomer,"_id",SYS(2015))
ADDPROPERTY(loCustomer,"FirstName","Rick")
ADDPROPERTY(loCustomer,"LastName","Strahl")
ADDPROPERTY(loCustomer,"Company","West Wind")
ADDPROPERTY(loCustomer,"Entered", DATETIME())loAddress = CREATEOBJECT("EMPTY")
ADDPROPERTY(loAddress,"Street","32 Kaiea")
ADDPROPERTY(loAddress,"City","Paia")
ADDPROPERTY(loCustomer,"Address",loAddress)loOrders = CREATEOBJECT("Collection")
ADDPROPERTY(loCustomer,"Orders",loOrders)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",121.10)
loOrders.Add(loOrder)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",44.22)
loOrders.Add(loOrder)
lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
loCustomer._id = SYS(2015)
loMongo.Save(loCustomer,"Customers")
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages inserted in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
which is pretty good, BUT most of the overhead in that is actually the FoxPro object to JSON conversion (all requests send JSON but the methods accept objects that are serialized to JSON).
Using JSON directly is a bit faster providing 10,000 records in only 3.5 seconds!
************************************************************************
* RunManyInsertsWithJson
****************************************
FUNCTION RunManyInsertsWithJson()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
TEXT TO lcJson TEXTMERGE NOSHOW
{
_id: "" ,
FirstName: "Rick",
LastName: "Strahl",
Company: "West Wind",
Entered: "< <TTOC(DATETIME(),3)> >",
Address: {
Street: "32 Kaiea",
City: "Paia"
},
Orders: [
{ OrderId: "ar431211", OrderTotal: 125.44, Date: "< <TTOC(DATETIME(),3)> >Z"},
{ OrderId: "fe134341", OrderTotal: 95.12, Date: "< <TTOC(DATETIME(),3)> >Z" }
]
}
ENDTEXTloMongo.Save(lcJson,"Customers",.T.)
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages updated in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
* RunManyInsertsWithJson
Although the latter is faster you have be very careful building JSON strings by hand as values have to be formatted properly. Strings have to be properly encoded and dates have to be new Date() formatted etc. so in general I hihgly recommend using FoxPro objects to ensure the serialization is proper.
+++ Rick ---
Looks great - Added 1000 records in 1.4 seconds. That's going to be good for me!.
If you haven't looked at NEO4J or another GRAPH database, they're really interesting. Gartner says that 25% of the Fortune 500 companies will be using it by 2016. It's a whole different way of analyzing data.
Thanks again for your help, und "Guten Rutsch ins Neue Jahr!"
Marty
You need to connect first. Look at the example:
https://github.com/RickStrahl/wwMongoDb#save-data-and-create-dbtable-if-it-doesnt-exist
If you use strings rather than objects or values, you have to pass the llIsJson parameter:
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/FoxMongoTest")
? loMongo.Save('{ firstname: "Rick" }',"Customers",.T.)
? loMongo.cErrorMsg
This will create the database and collection on first run.
I've updated the code on github. If you pull again, you should now get a Test.prg that you can just run without using FoxUnit.
+++ Rick ---
I now have the loMongo Object.
I tried this:
DO _config
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Save('{ firstname: "Rick" }',"Customers")
Got this error message.
OLE IDispatch exception code 0 from wwDotNetBridge: Method 'System.DBNull.SaveFromJson' not found...
I have to get back up to speed on Mongo. I'm guessing that "Customers" is a collection in the Test Database?
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
There is a error in wwMongoDB.save()
*** The reference to the Collection is Hard Coded:
*this.oLastResult = this.oBridge.InvokeMethod(this.oMongo,"SaveFromJson",lcJson,"Customers")
*** Should be:
this.oLastResult = this.oBridge.InvokeMethod(this.oMongo,"SaveFromJson",lcJson,lcCollection)
FUNCTION Save(lvEntity,lcCollection, llIsJson)
LOCAL lcResult, lcType, loSer, lcJsonIF IsNullOrEmpty(lvEntity)
this.SetError("Can't save null or empty entity")
RETURN .F.
ENDIFIF !llIsJson
lcJson = this.Serialize(lvEntity)
ELSE
lcJson = lvEntity
ENDIFthis.oBridge.cErrorMsg = ""
this.oLastResult = null
this.cLastRequest = lcJson
this.cLastResponse = lcJson
*** The reference to the Collection is Hard Coded:*this.oLastResult = this.oBridge.InvokeMethod(this.oMongo,"SaveFromJson",lcJson,"Customers")
*** Should be
this.oLastResult = this.oBridge.InvokeMethod(this.oMongo,"SaveFromJson",lcJson,lcCollection)
Marty,
I wouldn't recommend doing this.
Capturing the raw JSON is dangerous - anybody could send you whatever JSON they wanted and get that pushed straight into the database.
Here's what you should do:
- Capture the loAlbum object
- Validate it and make sure it has valid data in it
- Then save the object
There are more reasons for this than security as well. MongoDb doesn't work with 'regular' JSON - it uses a custom JSON dialect which requires special syntax for at least dates - stock JSON uses ISO formatted date strings, Mongo requires that dates uses either new Date() or ISODate syntax. The object conversion inside of the wwMongo object handles that for you but can't do that for straight JSON you pass as a string.
So - I highly recommend you use objects for saving - always, unless you have a specific reason not to do that.
+++ Rick ---
Now comes the exciting part - Modify the Music Store to use Mongo. I have to do some more checking into wwDotNetBridge, but I think what I need to do is:
what I want to do is:***********************************************************************
FUNCTION Album(loAlbum)
lcVerb = Request.GetHttpVerb()*** Handle alternate verbs with same URL with
*** separate method since we don't have method overloading in VFP
IF (lcVerb == "POST" OR lcVerb == "PUT")If lMongoIsLive
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/MusicStore")
lcJson = request.GetRawFormData()
loMongo.Save(lcJson,"Albums",.T.)
else
RETURN THIS.SaveAlbum(loAlbum)
endi
ENDIF
Yes, I just re-ran a small test that inserts 10,000 records in about 7.5 seconds:
************************************************************************
* RunManyInserts
****************************************
FUNCTION RunManyInserts()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")*** Recommend you assign your own ids for easier querying
ADDPROPERTY(loCustomer,"_id",SYS(2015))
ADDPROPERTY(loCustomer,"FirstName","Rick")
ADDPROPERTY(loCustomer,"LastName","Strahl")
ADDPROPERTY(loCustomer,"Company","West Wind")
ADDPROPERTY(loCustomer,"Entered", DATETIME())loAddress = CREATEOBJECT("EMPTY")
ADDPROPERTY(loAddress,"Street","32 Kaiea")
ADDPROPERTY(loAddress,"City","Paia")
ADDPROPERTY(loCustomer,"Address",loAddress)loOrders = CREATEOBJECT("Collection")
ADDPROPERTY(loCustomer,"Orders",loOrders)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",121.10)
loOrders.Add(loOrder)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",44.22)
loOrders.Add(loOrder)
lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
loCustomer._id = SYS(2015)
loMongo.Save(loCustomer,"Customers")
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages inserted in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
which is pretty good, BUT most of the overhead in that is actually the FoxPro object to JSON conversion (all requests send JSON but the methods accept objects that are serialized to JSON).
Using JSON directly is a bit faster providing 10,000 records in only 3.5 seconds!
************************************************************************
* RunManyInsertsWithJson
****************************************
FUNCTION RunManyInsertsWithJson()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
TEXT TO lcJson TEXTMERGE NOSHOW
{
_id: "" ,
FirstName: "Rick",
LastName: "Strahl",
Company: "West Wind",
Entered: "< <TTOC(DATETIME(),3)> >",
Address: {
Street: "32 Kaiea",
City: "Paia"
},
Orders: [
{ OrderId: "ar431211", OrderTotal: 125.44, Date: "< <TTOC(DATETIME(),3)> >Z"},
{ OrderId: "fe134341", OrderTotal: 95.12, Date: "< <TTOC(DATETIME(),3)> >Z" }
]
}
ENDTEXTloMongo.Save(lcJson,"Customers",.T.)
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages updated in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
* RunManyInsertsWithJson
Although the latter is faster you have be very careful building JSON strings by hand as values have to be formatted properly. Strings have to be properly encoded and dates have to be new Date() formatted etc. so in general I hihgly recommend using FoxPro objects to ensure the serialization is proper.
+++ Rick ---
Looks great - Added 1000 records in 1.4 seconds. That's going to be good for me!.
If you haven't looked at NEO4J or another GRAPH database, they're really interesting. Gartner says that 25% of the Fortune 500 companies will be using it by 2016. It's a whole different way of analyzing data.
Thanks again for your help, und "Guten Rutsch ins Neue Jahr!"
Marty
You need to connect first. Look at the example:
https://github.com/RickStrahl/wwMongoDb#save-data-and-create-dbtable-if-it-doesnt-exist
If you use strings rather than objects or values, you have to pass the llIsJson parameter:
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/FoxMongoTest")
? loMongo.Save('{ firstname: "Rick" }',"Customers",.T.)
? loMongo.cErrorMsg
This will create the database and collection on first run.
I've updated the code on github. If you pull again, you should now get a Test.prg that you can just run without using FoxUnit.
+++ Rick ---
I now have the loMongo Object.
I tried this:
DO _config
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Save('{ firstname: "Rick" }',"Customers")
Got this error message.
OLE IDispatch exception code 0 from wwDotNetBridge: Method 'System.DBNull.SaveFromJson' not found...
I have to get back up to speed on Mongo. I'm guessing that "Customers" is a collection in the Test Database?
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
Fixed.
+++ Rick ---
There is a error in wwMongoDB.save()
*** The reference to the Collection is Hard Coded:
*this.oLastResult = this.oBridge.InvokeMethod(this.oMongo,"SaveFromJson",lcJson,"Customers")
*** Should be:
this.oLastResult = this.oBridge.InvokeMethod(this.oMongo,"SaveFromJson",lcJson,lcCollection)
FUNCTION Save(lvEntity,lcCollection, llIsJson)
LOCAL lcResult, lcType, loSer, lcJsonIF IsNullOrEmpty(lvEntity)
this.SetError("Can't save null or empty entity")
RETURN .F.
ENDIFIF !llIsJson
lcJson = this.Serialize(lvEntity)
ELSE
lcJson = lvEntity
ENDIFthis.oBridge.cErrorMsg = ""
this.oLastResult = null
this.cLastRequest = lcJson
this.cLastResponse = lcJson
*** The reference to the Collection is Hard Coded:*this.oLastResult = this.oBridge.InvokeMethod(this.oMongo,"SaveFromJson",lcJson,"Customers")
*** Should be
this.oLastResult = this.oBridge.InvokeMethod(this.oMongo,"SaveFromJson",lcJson,lcCollection)
Marty,
I wouldn't recommend doing this.
Capturing the raw JSON is dangerous - anybody could send you whatever JSON they wanted and get that pushed straight into the database.
Here's what you should do:
- Capture the loAlbum object
- Validate it and make sure it has valid data in it
- Then save the object
There are more reasons for this than security as well. MongoDb doesn't work with 'regular' JSON - it uses a custom JSON dialect which requires special syntax for at least dates - stock JSON uses ISO formatted date strings, Mongo requires that dates uses either new Date() or ISODate syntax. The object conversion inside of the wwMongo object handles that for you but can't do that for straight JSON you pass as a string.
So - I highly recommend you use objects for saving - always, unless you have a specific reason not to do that.
+++ Rick ---
Now comes the exciting part - Modify the Music Store to use Mongo. I have to do some more checking into wwDotNetBridge, but I think what I need to do is:
what I want to do is:***********************************************************************
FUNCTION Album(loAlbum)
lcVerb = Request.GetHttpVerb()*** Handle alternate verbs with same URL with
*** separate method since we don't have method overloading in VFP
IF (lcVerb == "POST" OR lcVerb == "PUT")If lMongoIsLive
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/MusicStore")
lcJson = request.GetRawFormData()
loMongo.Save(lcJson,"Albums",.T.)
else
RETURN THIS.SaveAlbum(loAlbum)
endi
ENDIF
Yes, I just re-ran a small test that inserts 10,000 records in about 7.5 seconds:
************************************************************************
* RunManyInserts
****************************************
FUNCTION RunManyInserts()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")*** Recommend you assign your own ids for easier querying
ADDPROPERTY(loCustomer,"_id",SYS(2015))
ADDPROPERTY(loCustomer,"FirstName","Rick")
ADDPROPERTY(loCustomer,"LastName","Strahl")
ADDPROPERTY(loCustomer,"Company","West Wind")
ADDPROPERTY(loCustomer,"Entered", DATETIME())loAddress = CREATEOBJECT("EMPTY")
ADDPROPERTY(loAddress,"Street","32 Kaiea")
ADDPROPERTY(loAddress,"City","Paia")
ADDPROPERTY(loCustomer,"Address",loAddress)loOrders = CREATEOBJECT("Collection")
ADDPROPERTY(loCustomer,"Orders",loOrders)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",121.10)
loOrders.Add(loOrder)loOrder = CREATEOBJECT("Empty")
ADDPROPERTY(loOrder,"Date",DATETIME())
ADDPROPERTY(loOrder,"OrderId",SUBSTR(SYS(2015),2))
ADDPROPERTY(loOrder,"OrderTotal",44.22)
loOrders.Add(loOrder)
lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
loCustomer._id = SYS(2015)
loMongo.Save(loCustomer,"Customers")
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages inserted in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
which is pretty good, BUT most of the overhead in that is actually the FoxPro object to JSON conversion (all requests send JSON but the methods accept objects that are serialized to JSON).
Using JSON directly is a bit faster providing 10,000 records in only 3.5 seconds!
************************************************************************
* RunManyInsertsWithJson
****************************************
FUNCTION RunManyInsertsWithJson()loMongo = this.CreateMongo()
*** Note objects are serialized as lower case
loCustomer = CREATEOBJECT("EMPTY")lnCount = 10000
lnStart = SECONDS()
FOR lnX = 1 TO lnCount
TEXT TO lcJson TEXTMERGE NOSHOW
{
_id: "" ,
FirstName: "Rick",
LastName: "Strahl",
Company: "West Wind",
Entered: "< <TTOC(DATETIME(),3)> >",
Address: {
Street: "32 Kaiea",
City: "Paia"
},
Orders: [
{ OrderId: "ar431211", OrderTotal: 125.44, Date: "< <TTOC(DATETIME(),3)> >Z"},
{ OrderId: "fe134341", OrderTotal: 95.12, Date: "< <TTOC(DATETIME(),3)> >Z" }
]
}
ENDTEXTloMongo.Save(lcJson,"Customers",.T.)
ENDFORthis.MessageOut(TRANSFORM(lnCount) + " messages updated in " + TRANSFORM(SECONDS() - lnStart))
ENDFUNC
* RunManyInsertsWithJson
Although the latter is faster you have be very careful building JSON strings by hand as values have to be formatted properly. Strings have to be properly encoded and dates have to be new Date() formatted etc. so in general I hihgly recommend using FoxPro objects to ensure the serialization is proper.
+++ Rick ---
Looks great - Added 1000 records in 1.4 seconds. That's going to be good for me!.
If you haven't looked at NEO4J or another GRAPH database, they're really interesting. Gartner says that 25% of the Fortune 500 companies will be using it by 2016. It's a whole different way of analyzing data.
Thanks again for your help, und "Guten Rutsch ins Neue Jahr!"
Marty
You need to connect first. Look at the example:
https://github.com/RickStrahl/wwMongoDb#save-data-and-create-dbtable-if-it-doesnt-exist
If you use strings rather than objects or values, you have to pass the llIsJson parameter:
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Connect("mongodb://localhost/FoxMongoTest")
? loMongo.Save('{ firstname: "Rick" }',"Customers",.T.)
? loMongo.cErrorMsg
This will create the database and collection on first run.
I've updated the code on github. If you pull again, you should now get a Test.prg that you can just run without using FoxUnit.
+++ Rick ---
I now have the loMongo Object.
I tried this:
DO _config
DO wwmongodb
loMongo = CREATEOBJECT("wwMongoDb")
loMongo.Save('{ firstname: "Rick" }',"Customers")
Got this error message.
OLE IDispatch exception code 0 from wwDotNetBridge: Method 'System.DBNull.SaveFromJson' not found...
I have to get back up to speed on Mongo. I'm guessing that "Customers" is a collection in the Test Database?
Ok, so tried this out from the sample and see the same error.
Looks like the /bin folder is missing in the /Fox folder - .gitignor snafu. For now, you can copy the /bin folder from /dist/bin into /fox and then it should work.
I'll update the code on Github.
+++ Rick ---
I unblocked all the .dll files, but I'm getting this error.
Could not load file or assembly 'Westwind.Data.MongoDb.dll' or one of its dependencies. The system cannot find the file specified.
I tried moving all the files into the same directory but without success.
Marty,
I spent a few hours putting together a prototype for accessing MongoDb in a new library, wwMongoDb. You can check it out here from GitHub:
wwMongoDb - A small library to provide based data access to MongoDb from Visual FoxPro
It provides basic access to read and write data in Mongo. You can do the following:
- Save Entities to Mongo
- Run Find queries to select Collections of data using Mongo shell query syntax
- Run FindOne/Load queries to select individual Entities
It works well, and it's surprisingly FAST (especially inserts). Even queries do well, but I haven't tested with large data sets.
This is definitely a very small subset of featuers, but this is enough to allow you to push data into Mongo and get it back out again. There's a lot more that could be done. It needs more work for things like group operations and additional query operations like sorts and admin features.
But this is prototype that gets you started.
Let me know what you think...
+++ Rick ---
Rick,
I'm trying to optimize a VFP app for conversion to .NET. One of the features is a Survey tool which invariably has a different schema for every instance. In VFP, I generate a unique new table for every Survey, but that won't fly in SQL server. I've got to store the data as XML in a varchar or as JSON in MongoDB.
I have been able to make the data store work in Mongo using NODE as a proof of concept, but I can't convert the whole app to NODE or .NET. The conversion to .NET is being done by another team - but it's probably over a year or more away.
Speed is not really an issue.
Thanks for the responese,
Marty
Using MongoDb from FoxPro would probably be inhibitively slow because there's no good way to map the data to anything useful. We'd have to use FoxPro based JSON parsing to parse the data to returned from results to FoxPro objects which is fairly slow. Mongo also doesn't have a direct REST API - it uses a proprieatary binary format for the interface, so you'd have to interop with something that does support it (like .NET). The latter might work, but I'm not suer if you can actually send JSON queries directly as strings.
IAC, I'm not sure why you'd want to do this... wouldn't it be easier to just use Node or .NET directly to access the data where the drivers are natively supported?
It can be done, but it's not exactly a trivial process and you'd have to at the very least build your queries using JSON strings.
+++ Rick ---
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
I tried Regex also.. still didnt work.. is there a way to do this?
Thanks
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
We're limited by what the C# provider provides access to.
If there's enough interest I'll keep building this up a bit.
Several people actually have offered to put in 'sponsorship' for this project which is kind of an interesting proposition.
I guess we can all thank Marty for getting this ball of wax rolling - without him I would have never even looked at this, but given how well it does work with just the basics in place I'm starting to change my tune. I use MongoDb on a lot of stuff from .NET these days and it's wonderful as it relieves many issues having to deal with related data. However, I haven't done a lot with heavily report centric data that needs to jump collections - that seems to be the weak spot with NoSql in general.
Be curious to hear what others come up with.
+++ Rick ---
I downloaded the samples on github... Worked perfect... added tons of recs.. .find tons of recs.. However, wildcard doent not work... loMongo.Find([{ pn: "NT*" }],"somefile")
I tried Regex also.. still didnt work.. is there a way to do this?
Thanks
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
RegEx works for sure - I use it for several tests:
loCustomers = loMongo.Find('{ firstname: /^R.*/i, entered: { "$gte": new Date(2011,11,1) } }',;
"Customers",0,100 ) && skip 0, limit 100
You should also be able to do something like this:
db.Customers.find( { 'firstname': {$gte: "M",$lt:"N" }})
But AFAIK there's no 'wildcard' syntax with MongoDb's BSON query operators.
+++ Rick ---
I downloaded the samples on github... Worked perfect... added tons of recs.. .find tons of recs.. However, wildcard doent not work... loMongo.Find([{ pn: "NT*" }],"somefile")
I tried Regex also.. still didnt work.. is there a way to do this?
Thanks
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
Thanks
Jim
Jim,
RegEx works for sure - I use it for several tests:
loCustomers = loMongo.Find('{ firstname: /^R.*/i, entered: { "$gte": new Date(2011,11,1) } }',;
"Customers",0,100 ) && skip 0, limit 100
You should also be able to do something like this:
db.Customers.find( { 'firstname': {$gte: "M",$lt:"N" }})
But AFAIK there's no 'wildcard' syntax with MongoDb's BSON query operators.
+++ Rick ---
I downloaded the samples on github... Worked perfect... added tons of recs.. .find tons of recs.. However, wildcard doent not work... loMongo.Find([{ pn: "NT*" }],"somefile")
I tried Regex also.. still didnt work.. is there a way to do this?
Thanks
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
Thanks Rick..
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
You really should be building business objects that return the appropriate view data to the client. Otherwise you're accessing the data directly which IMHO is not a good way to do data access in a client application - too much of a security issue. You need some intermediary to handle security for the applicaiton, validation, and data filtering plus - most likely - the abstraction for the actual queries to run. Direct data access against he database from the client will just lead to all sorts of pain later on...
But you're definitely right - there's a bit of overhead in converting data to FoxPro and then converting back. There are generic front ends that you can run that can act as a 'data server' in PHP, Python etc. The goal should be to minimize the amount of data that is returned so that requests stay quick. The music store does some very inefficeint stuff actually (like a one to n traversal for albums in order to load up the business objects properly) but that can be solved with aggregations etc.
This is one of the reasons I mentioned originally that using MongoDb from FoxPro is not necessarily the most effective way to access data. if you absolutely need Mongo from FoxPro it's a good way to go, but I'm not sure if it's a good db access choice if other options like using a SQL backend are an option.
Nativly MongoDb uses a custom JSON dialect (BSON) and you need some intermediary to convert things. There's a built in REST service, but it's very limited.
+++ Rick ---
I've been playing with the mongodb stuff.. Is there a way to just return raw json from the mongo server? I know I can iterate the returned object.. but that is too much overhead... something like mongo does..db.table.find() .. then I can just send the resulting json back to the client/browser.....
Thanks Rick..
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
I see there are some Mongodb ODBC drivers out there that work with VFP... Excel.. Word... what is your opinion on this?
Thanks
Jim
Nope, nor would I recommend this.
You really should be building business objects that return the appropriate view data to the client. Otherwise you're accessing the data directly which IMHO is not a good way to do data access in a client application - too much of a security issue. You need some intermediary to handle security for the applicaiton, validation, and data filtering plus - most likely - the abstraction for the actual queries to run. Direct data access against he database from the client will just lead to all sorts of pain later on...
But you're definitely right - there's a bit of overhead in converting data to FoxPro and then converting back. There are generic front ends that you can run that can act as a 'data server' in PHP, Python etc. The goal should be to minimize the amount of data that is returned so that requests stay quick. The music store does some very inefficeint stuff actually (like a one to n traversal for albums in order to load up the business objects properly) but that can be solved with aggregations etc.
This is one of the reasons I mentioned originally that using MongoDb from FoxPro is not necessarily the most effective way to access data. if you absolutely need Mongo from FoxPro it's a good way to go, but I'm not sure if it's a good db access choice if other options like using a SQL backend are an option.
Nativly MongoDb uses a custom JSON dialect (BSON) and you need some intermediary to convert things. There's a built in REST service, but it's very limited.
+++ Rick ---
I've been playing with the mongodb stuff.. Is there a way to just return raw json from the mongo server? I know I can iterate the returned object.. but that is too much overhead... something like mongo does..db.table.find() .. then I can just send the resulting json back to the client/browser.....
Thanks Rick..
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
You just have to be careful with JSON that comes directly out of MongoDb because it's a custom flavor of JSON where dates, ids and a few other things are not using standard JavaScript values, but use functions that won't parse with plain JS JSON parser. So directly feeding the results out of a MongoDb query likely wouldn't work well anyway - you always need some sort of intermediary and I think there are a number of solutions available for this for Node and other platforms. It'd be easy to build something like this for .NET as well given the tools I created actually as all that could be done is just do string pass throuhghs...
+++ Rick ---
Thanks Rick... If I need speed.. i'll point the client/url to our NodeJS server... but for important things... getting the data from mongodb via vfp and running through a business object is probably better than doing it in the browser..... So if I used this in wconnect... I should copy the wwdotnetbridge.dll over the one in \wconnect ... what about all the classes.. wwapi.. wwutils.. etc? Should I overwrite these too?
I see there are some Mongodb ODBC drivers out there that work with VFP... Excel.. Word... what is your opinion on this?
Thanks
Jim
Nope, nor would I recommend this.
You really should be building business objects that return the appropriate view data to the client. Otherwise you're accessing the data directly which IMHO is not a good way to do data access in a client application - too much of a security issue. You need some intermediary to handle security for the applicaiton, validation, and data filtering plus - most likely - the abstraction for the actual queries to run. Direct data access against he database from the client will just lead to all sorts of pain later on...
But you're definitely right - there's a bit of overhead in converting data to FoxPro and then converting back. There are generic front ends that you can run that can act as a 'data server' in PHP, Python etc. The goal should be to minimize the amount of data that is returned so that requests stay quick. The music store does some very inefficeint stuff actually (like a one to n traversal for albums in order to load up the business objects properly) but that can be solved with aggregations etc.
This is one of the reasons I mentioned originally that using MongoDb from FoxPro is not necessarily the most effective way to access data. if you absolutely need Mongo from FoxPro it's a good way to go, but I'm not sure if it's a good db access choice if other options like using a SQL backend are an option.
Nativly MongoDb uses a custom JSON dialect (BSON) and you need some intermediary to convert things. There's a built in REST service, but it's very limited.
+++ Rick ---
I've been playing with the mongodb stuff.. Is there a way to just return raw json from the mongo server? I know I can iterate the returned object.. but that is too much overhead... something like mongo does..db.table.find() .. then I can just send the resulting json back to the client/browser.....
Thanks Rick..
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui
Thanks again Rick!!!
You just have to be careful with JSON that comes directly out of MongoDb because it's a custom flavor of JSON where dates, ids and a few other things are not using standard JavaScript values, but use functions that won't parse with plain JS JSON parser. So directly feeding the results out of a MongoDb query likely wouldn't work well anyway - you always need some sort of intermediary and I think there are a number of solutions available for this for Node and other platforms. It'd be easy to build something like this for .NET as well given the tools I created actually as all that could be done is just do string pass throuhghs...
+++ Rick ---
Thanks Rick... If I need speed.. i'll point the client/url to our NodeJS server... but for important things... getting the data from mongodb via vfp and running through a business object is probably better than doing it in the browser..... So if I used this in wconnect... I should copy the wwdotnetbridge.dll over the one in \wconnect ... what about all the classes.. wwapi.. wwutils.. etc? Should I overwrite these too?
I see there are some Mongodb ODBC drivers out there that work with VFP... Excel.. Word... what is your opinion on this?
Thanks
Jim
Nope, nor would I recommend this.
You really should be building business objects that return the appropriate view data to the client. Otherwise you're accessing the data directly which IMHO is not a good way to do data access in a client application - too much of a security issue. You need some intermediary to handle security for the applicaiton, validation, and data filtering plus - most likely - the abstraction for the actual queries to run. Direct data access against he database from the client will just lead to all sorts of pain later on...
But you're definitely right - there's a bit of overhead in converting data to FoxPro and then converting back. There are generic front ends that you can run that can act as a 'data server' in PHP, Python etc. The goal should be to minimize the amount of data that is returned so that requests stay quick. The music store does some very inefficeint stuff actually (like a one to n traversal for albums in order to load up the business objects properly) but that can be solved with aggregations etc.
This is one of the reasons I mentioned originally that using MongoDb from FoxPro is not necessarily the most effective way to access data. if you absolutely need Mongo from FoxPro it's a good way to go, but I'm not sure if it's a good db access choice if other options like using a SQL backend are an option.
Nativly MongoDb uses a custom JSON dialect (BSON) and you need some intermediary to convert things. There's a built in REST service, but it's very limited.
+++ Rick ---
I've been playing with the mongodb stuff.. Is there a way to just return raw json from the mongo server? I know I can iterate the returned object.. but that is too much overhead... something like mongo does..db.table.find() .. then I can just send the resulting json back to the client/browser.....
Thanks Rick..
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
Please let me know if you run into any issues.
+++ Rick ---
Thanks Rick.... I actually have it working with wconnect... mainly for static lists and lookup of data from a mongodb collection with over 3 million docs...
Thanks again Rick!!!
You just have to be careful with JSON that comes directly out of MongoDb because it's a custom flavor of JSON where dates, ids and a few other things are not using standard JavaScript values, but use functions that won't parse with plain JS JSON parser. So directly feeding the results out of a MongoDb query likely wouldn't work well anyway - you always need some sort of intermediary and I think there are a number of solutions available for this for Node and other platforms. It'd be easy to build something like this for .NET as well given the tools I created actually as all that could be done is just do string pass throuhghs...
+++ Rick ---
Thanks Rick... If I need speed.. i'll point the client/url to our NodeJS server... but for important things... getting the data from mongodb via vfp and running through a business object is probably better than doing it in the browser..... So if I used this in wconnect... I should copy the wwdotnetbridge.dll over the one in \wconnect ... what about all the classes.. wwapi.. wwutils.. etc? Should I overwrite these too?
I see there are some Mongodb ODBC drivers out there that work with VFP... Excel.. Word... what is your opinion on this?
Thanks
Jim
Nope, nor would I recommend this.
You really should be building business objects that return the appropriate view data to the client. Otherwise you're accessing the data directly which IMHO is not a good way to do data access in a client application - too much of a security issue. You need some intermediary to handle security for the applicaiton, validation, and data filtering plus - most likely - the abstraction for the actual queries to run. Direct data access against he database from the client will just lead to all sorts of pain later on...
But you're definitely right - there's a bit of overhead in converting data to FoxPro and then converting back. There are generic front ends that you can run that can act as a 'data server' in PHP, Python etc. The goal should be to minimize the amount of data that is returned so that requests stay quick. The music store does some very inefficeint stuff actually (like a one to n traversal for albums in order to load up the business objects properly) but that can be solved with aggregations etc.
This is one of the reasons I mentioned originally that using MongoDb from FoxPro is not necessarily the most effective way to access data. if you absolutely need Mongo from FoxPro it's a good way to go, but I'm not sure if it's a good db access choice if other options like using a SQL backend are an option.
Nativly MongoDb uses a custom JSON dialect (BSON) and you need some intermediary to convert things. There's a built in REST service, but it's very limited.
+++ Rick ---
I've been playing with the mongodb stuff.. Is there a way to just return raw json from the mongo server? I know I can iterate the returned object.. but that is too much overhead... something like mongo does..db.table.find() .. then I can just send the resulting json back to the client/browser.....
Thanks Rick..
Jim
I'd be in too... I use arrays in 1 object in mongodb for each document... The array is easy to iterate, and I believe faster... I'm no expert in mongodb.. but love the speed and how it scales... Our company is thinking about using Mongodb to store all the data... and use vfp to query the data and do the business logic...
I'd be willing to pay for this to be part of wconnect!!!!
Thanks
Jim
Rick,
This was just what I need to get Angular under control. Trying to learn Node and Express all at the same time was confusing.
Rick, do you feel like extending VFP to MongoDB and NEO4J?
If $500 makes it worthwhile, I'm in again.
Thanks,
Marty
West Wind Technologies
Making waves on the Web
from Maui