Products

Solutions

Resources

Partners

Community

About

New Community Website

Ordinarily, you'd be at the right spot, but we've recently launched a brand new community website... For the community, by the community.

Yay... Take Me to the Community!

Welcome to the DNN Community Forums, your preferred source of online community support for all things related to DNN.
In order to participate you must be a registered DNNizen

HomeHomeDNN Open Source...DNN Open Source...Module ForumsModule ForumsMapMapMap not pulling all users from DNN user databaseMap not pulling all users from DNN user database
Previous
 
Next
New Post
3/20/2008 1:54 AM
 

So I have the beta installed, and wow, it is very cool. Nicely done.

I have it set up as a directory layout and have Users online as the data source.

When I execute user import, it successfully imports some of the users, but not all. All of the ones that are imported do successfully plot. I'm trying to track down why only a certain set is imported. All user accounts require a valid profile for login, so they are mostly normalized, at least they all have the required fields.

What would a good debugging process be to track this down so that I can get all users importing and plotted?

 
New Post
3/20/2008 11:35 AM
 
I had my brother in law, a .NET developer look at the problem, and here is his response:

here is how I currently understand it to work from looking at the code and the database:

each time you view the map it triggers (on a delay) the same thing as clicking the "Execute User Import" button on the DataSource admin section so its safe to click the button whenever you like. the "Execute Geo Coding" doesn't realy do anything in the mode you are currently in as that code is triggered with the "Execute User Import" already.
So because "Execute User Import" gets triggered as the map is viewed that is why the map should keep it self up to date in the future without having to do anything.

when "Execute User Import" runs it tries to lookup a persons address in in google maps to get the long/lat coordinates. if it fails then it will not enter it into the data list. It is esentialy skiped. So the first thing was that I found 2 addresses that didn't get long/lat coords from google so they were skiped. So thats 1 bug. but that doesn't account for the other 15 or so that didn't make it. the next bug revolves around a timming issue of some sort. I can't actualy tell from the code but it looks like it might be a connection string time out on the database or max open sockets when looking up the long/lat on google. something like that. Either way its throwing an exception that isn't handle properly. I can't tell without seeing more of the setting on the server but the end result it that it will import about 10 records then fail the rest of them. Now this normaly wouldn't be a problem because you would think that it will just try the others latter and get them all but it doesn't. the code tries onces then never tries those records again. All the user records have an ID that auto-incriments +1. the database stores the highest ID from the last time it ran "Execute User Import" so it knows where to pick up where it left off and not try importing users its already tried again.
So what I did was to change the ID back to the last successful import, then run the import again, and again. this was my shake-it method that eventualy worked. There was one other bug I ran across. one of the records refused to import and would even refuse the last ID incrimentor to incriment so it got stuck. I couldn't fugure out why from the code and since there is no exception log, there is no way to see what is happening. this one I had to skip by placing the last ID just one ID above the one that wouldn't import. then I looked up the lat/long by hand and entered the record myself.

hopefuly these bugs will get fixed in the future release. but I do believe that it should work OK for you either way moving forward. this is because the system will natualy import 1 user at a time (as they register on the site). Not 30 at a time (like the initial import) So hopefuly the timeout issue, or socket issue, or whatever it is won't effect you. So it should work. I would test it a few times by creating an account and seeing if the map updates. then you can delete the account. (also take note that it apears that users will not be deleted from the map if an account is deleted so you will have to do this manualy.)

so here is how you do what I did if you need to shake users into the list again.

SQL:
select u.UserID, u.UserName, u.FirstName, u.LastName, Guid as IsOnMap from dnn_users u right outer join DNN_Map_Points m on u.UserID = m.Guid

this shows you a list of all users. the last field (IsOnMap) will be blank if there isn't a record for them on the map.
we will refer to this query as "IsOnMap"

SQL:
select serviceFlag from DNN_Map_Sources where sourceid = 1

this shows you the last UserID that the "Execute User Import" will pick up from next time it is ran. (it will usualy be the highest UserID in the list (currently its 36)


SQL:
update DNN_Map_Sources set serviceflag = 26 where sourceID = 1

this will update the last UserID that the "Execute User Import" will start from to be whatever you want. (just change 26 to whatever number)

URL:
http://maps.google.com/maps/geo?output=xml&key=[key goes here]&q=

this URL will return the long/lat from google maps and is the same url used in the code. you can use it to get any long/lat for future records that the system has trouble getting itself (I ran into 1 I had to do by hand) Just add the address to the end of the query (after &q=). Its so cool it feels magical.

OK. here is how you shake it:
if you were to clear out the records ("Delete all point") and re-import ("Execute User Import") you would get about 10 record.
run the "IsOnMap" query and find the last UserID that was successfuly imported and then execute the update query change the last UserID to be the last *successful* UserID
Then run "Execute User Import" again.
repeate until there all done.
if it gets stuck on a record, you can verify it by runnig the "select service flag..." query to see what record its picking up from. if its the same one every time then skip it by running the update query with an ID just 1 greater then its current id.
then you will have to add the record manualy by hand.
to do that, get the users address and punch it into the google URL to get your long/lat code. then in the data section of the map click "new" and add the record. (the "timer" field should be the UserID)
 
New Post
3/20/2008 1:10 PM
 

Another quip from my bro-in-law:

 

The query that is pulling the users is correct.  It is getting *all* the users.

The bug is when it processes those users.  If it doesn’t process right, then it doesn’t insert it onto the map

 
New Post
3/23/2008 9:02 PM
 

Yes, these statements are correct for the most part. Although, in testing there has not been a timeout issue that I've come across. Essentially what is handle with users is a forward only processing of addresses. The failure to get the correct points for those users is normally dependant on the format of the address coming in. Google can at times be a bit annoying  with the processing of the points that are requested, but it also occasionally responds with messages that were not anticipated. I've found in tests that is appears to be about 80% effective.

There are two items noted here that I am working through, which were caused due to some specific requirements on the system.

1. In GoMap, we used a service to keep the points up to date for the users. This service ran on a schedule to keep everything synced. I could not use a service for the core map.

2. In GoMap, the user updates triggered flags in the DB for resetting the coordinates. I could not use a trigger in the database for the core map.

So, due to these constraints, the handling of Geocoding, and sourcing the users is handled in a single forward pass and possible callback geocoding request triggered by user requests (The rule here is simple, every 5 minutes - if a single user request, is left to run for more than 30 minutes, a ClientAPI callback is raised to Geocode the points.

The points are geocoded in blocks of 300 (i think off hand here), and those are attempted until the request times out or the locations are obtained, whichever comes first. Clicking the link to import the users does the same thing.

I am working on some additional ties into the system to make it a bit more intelligent, but without the triggers and events that Gomap utilized, it is a bit more difficult.

Please feel free provide any future items you would like to see in the map - I love the feedback!

Thanks!



Kevin M Schreiner
[P] 410.327.0007 [F] 410.327.4082 | info@r2integrated.com
 
Previous
 
Next
HomeHomeDNN Open Source...DNN Open Source...Module ForumsModule ForumsMapMapMap not pulling all users from DNN user databaseMap not pulling all users from DNN user database


These Forums are dedicated to discussion of DNN Platform and Evoq Solutions.

For the benefit of the community and to protect the integrity of the ecosystem, please observe the following posting guidelines:

  1. No Advertising. This includes promotion of commercial and non-commercial products or services which are not directly related to DNN.
  2. No vendor trolling / poaching. If someone posts about a vendor issue, allow the vendor or other customers to respond. Any post that looks like trolling / poaching will be removed.
  3. Discussion or promotion of DNN Platform product releases under a different brand name are strictly prohibited.
  4. No Flaming or Trolling.
  5. No Profanity, Racism, or Prejudice.
  6. Site Moderators have the final word on approving / removing a thread or post or comment.
  7. English language posting only, please.
What is Liquid Content?
Find Out
What is Liquid Content?
Find Out
What is Liquid Content?
Find Out