Using a C++ DLL in Cube

October 10th, 2010

One thing that can drastically speed Cube is using a DLL to do big tasks, like Nested Logit Mode Choice. However, doing this can be fraught with hair-pulling errors.  This post shows some techniques to keep your hair on your head.  This post is written for a travel demand modeler, not a computer science person!

RTFM (Read The Fine Manual)

Read the help file for Matrix CALL statement.  The struct statement is pretty important, and the sprintf lines will be used throughout.

Memory Pointers

One of the most important things to understand is that because there are so many variables that can be passed between Cube and the C++ DLL, the memory pointers are passed instead.  Also, one of those "pull your hair out" things relates to this - if you attempt to access a memory pointer that hasn't been initialized, you get a crash that gives no error.

Because of this, the variables in the struct statement have a *, which notes that it is a memory pointer.

To keep from getting the crash-with-no-error, the following statement works well to test and allows a default to be used if the variable 'MarketSegments' is not set in Cube.

int MarketSegments=4;


Matrix In, Matrix Out

While the Help file says that you can get to defined working matrixes using

static double **MW;

I can't get it to work using C++ (I have gotten it to work in C).  Instead, use the following:

static double **MW=NULL;

This will enable you to use MW[m][j] (where m is the MW number, and j is the j-zone).

You can also set the MW variables, but it does NOTHING if you don't set the MW to something in Cube Voyager.  Ergo, if you set


Your output will be 0 unless you do the following in Cube Voyager


Array Variables

One of the tricks I use to get array variables out of Cube is this

float ArrayVariable[7]={0,0,0,0,0,0,0};  //Note: I'm using 1-6.  Setting this to 7 means 0-6.  Setting it to 6 would mean 0-5
double* tmpAV=NULL;
for(int x=1;x<=6;x++)

This code above checks that the ArrayVariable, fills them into a temporary variable, and then sets the actual variable.

Compilation Linker Settings

When compiling, you need to set the EXPORT flag so the name is predictable and correct.  To do this, go to your project's property pages - Configuration Properties - Linker - Command Line.  You need to add "/EXPORT:FunctionName" under Additional Options.  See the screenshot below


Other Weird Stuff

Any error in C++ that does not cause a compilation error results in one of those useless "this program has an error and will be closed" and crashes Task Monitor.  That being said, write messages to the output file frequently (if at least during debugging).  This can assist with finding typos (like, say, %10.65f in an sprintf statement, which means 65 decimal places in a 10-width line).

Cube Voyager: Using Cluster with DBI

October 3rd, 2010

Credit for this goes to Citilabs Support, although I made some adaptations.

In Matrix when using DBI, PAR ZONES=1 will effectively shut off Cluster. Therefore, the following works really well.



start = (I-1)*recs+1
end = I*recs

LOOP _r=start,end

This script sets each core to process an equal portion of the database with any remainder (e.g if you cluster 4 records over 3 cores) to the last core.

Cube Voyager Speed Clinic

September 26th, 2010

There are several issues with long travel demand model run times.  Deep down, these are supposed to be planning tools, and taking too long for results can reduce the practicality of using a travel demand model in decision making.

In Cube Voyager, I've been finding more ways to cut runtimes down as much as possible, and below is the list with some of the rationale.

Keep JLoops at a Minimum

The Matrix program runs in an implied ILOOP.  This being the case, anything in the script runs many times (as many as the zones you have).  Using a JLOOP in a 2,000 zone model means that there are  4,000,000 calculations to be done for each COMP statement.  What happens if you have 3 COMP statements?  You go from 4,000,000 to 12,000,000 calculations.  This is even worse if the calculations include a lookup function, which are generally slow.

Keep Files Small - Only Write What You Have To

This is a no-brainer.  The more that Cube (or anything, for that matter) has to write to disk, the longer the runtime.

Replace RECI with DBI wherever possible

DBI can be clustered (look for a post on that in the coming weeks).  While I'm not sure if there is any difference on one core, being able to use Cluster is the trump card.

Use Cluster Dynamically and Wherever Possible

Standardize the Cluster ID in your code, but set the process list to a catalog key as with below:


Using Cluster to your advantage is critical to having a fast model.

Blackberry + GPSed + Lightroom + Lightroom Plugin = Awesome!

July 24th, 2010

For those of us using older digital cameras that do not have GPS capabilities (like my trusted D70), there is a way to use a Blackberry to capture the GPS coordinates and use a Lightroom plugin to put the GPS coordinates into the EXIF Metadata.  Once the metadata is written, sites like Panoramio and Flickr will recognize the location.

The first step to this is to have a Blackberry.  I'm sure there is a way to do this with an Android based phone or an iPhone, but I haven't used them, so I don't know how to do this on those.

On your Blackberry, open App World and search for, download, and install GPSed.  This program can be used to track your location.  BIG IMPORTANT NOTE: this will use much more battery than normal.  The amount of battery is used WAY MORE if you go into an area with no service.

Before you go 'Shooting'

First off, set the time on your camera.  It will make your life easier down the road.

Then, at the start of your shoot/photowalk, open GPSed and select New Track... in the menu:


And give it a name:

Start shooting!  While you are shooting, the GPSed free version log a GPS point roughly every 2 seconds.  This can be affected by a number of things,like tall buildings, clouds, trees, and having your Blackberry in your pocket or a case.  While those do create problems, leaving my Blackberry in its case while I walk through downtown is pretty good, but will sometimes miss points.

When you are done...

When you are done, open the menu ans select Finish Track.  On the following screen, it will ask you if you want to share or upload the track.  I tend to use "Do Nothing".


After the track is saved (and uploaded or shared if you did that), you will need to convert the track to GPX format.

The next steps on the Blackberry can be done now or at home.  I tend to do this part at home, since it goes hand-in-hand with the rest of the process.

At Home...

In GPSed, click the menu and select Pages > Track List.  You will want to select your track, click the menu, and select Convert to GPX...

-> ->


It will take a few to process.  My ~2 hour, 3 mile photowalk around 15-30 seconds.  Make sure you remember where it tells you it put the file!


After this, you can close GPSed.  You will want to locate that file and transfer it to your computer.  I used email.

-> ->

On Your Computer...

At this point, all of the Blackberry steps are completed.  You should be able to open your email and get the GPX file to process in Lightroom.  Save this somewhere.  I use /Users/andrew/Pictures/Geoencoding on my Mac.

For this part, you will need Jeffrey Friedl's GPS Plugin.  If you have a lot of pictures, you will want to donate to him to remove the 10-pic-at-a-time block.  Also, if you do this more than once or twice in your life, you should donate to him 🙂

To set the GPS coordinates on the actual pictures, open Lightroom and import your pictures.  Also, use the Lightroom Plugin Manager to open the GPS plugin.

Select all the images you want to set GPS coordinates to (CMD-A on Mac, Cntrl-A on PC, if you have them in the same folder).  Then, go to File-Plugin Extras - Geoencode...

There are several important things on the resulting window...

The important stuff here:

  • Make sure the tracklog tab is selected.
  • Make sure you've selected the GPX file that you emailed to yourself.
  • Select UTC as the timezone - this is because the times from the GPS satellites are in UTC.
  • I've had the most luck with a 30 second fuzzyness.  Increase if you are in a downtown, decrease if you are not and you were running.
  • If your camera doesn't have the correct time, correct it!  You really shouldn't have to use much in the camera time correction.

Once you click Geoencode, the GPS information will be added to all the pictures it can.  Congratulations, your pictures are mappable!  Now, if you use Jeffery Friedl's Flicker plugin, Flickr will know where to place them on the map!

Tour-Based Modeling: Why is it Important?

June 12th, 2010

One thing that is constantly bounced around is why tour-based modeling is better than trip based modeling.  We've been using trip based modeling for 50 years, isn't it timeless?


Fifty years ago, when the trip based modeling methodologies were developed, the primary reason was to evaluate highway improvements.  While tolling was in use, the bonding requirements were likely different.  Transit, while extremely important, was not in the public realm (the streetcars were normally privately owned by the area's electric company).

Now, there are a lot of demands on travel models:

  • Tolling/Toll Road analysis at a better level
  • Different tolling schemes (area tolling, cordon tolling)
  • Travel Demand Management (telecommuting, flex hours, flex time, alternative schedules)
  • Better freight modeling (which now is becoming commodity flow and commercial vehicle modeling)
  • Varying levels of transit (local bus, express bus, intercity bus, BRT, light rail, and commuter rail

While many of these can be done with trip based models, most of them cannot be done well with trip based models.  There are a number of reasons, but the few that come to mind are aggregation bias, modal inconsistency, and household interrelationships.

Aggregation Bias

Aggregation bias occurs when averages are used to determine an outcome.  For example, using a zonal average vehicles per household, you miss the components that form the average, such as:

20 households, average VPHH = 2.2
2 HH VPHH = 0
5 HH VPHH = 1
4 HH VPHH = 2
6 HH VPHH = 3
3 HH VPHH = 4+

The trip generation and modal choices (car, bus, bike, walk, etc.) among these households are all different, and are even more more different if you look at the number of workers per household.

Modal Inconsistency

In trip based modeling, "people" are not tracked throughout their day.  So, if someone rides the bus to work, there is nothing in the model to ensure that they don't drive from work to get lunch.  While we don't want to force people to use the same mode, since many people will use the bus to get to work and then walk to lunch or to go shopping during lunch, we want to make sure that there is some compatibility of modes.

Household Interrelationships

One of the features of of tour based models is determining each person's daily activity pattern.  During this process, certain person types can determine what another person is doing.  For example, if a preschool age child is staying home, an adult (whether they are a worker or not) HAS to stay home.  Another example is if a school-non-driving-age child is going on a non-mandatory trip, an adult must accompany them.  Trip based models don't know about the household makeup and the household interaction.

The above are only three of the many reasons why tour-based modeling is important.  There are many more, but I feel these are some of the most important and some of the easiest to understand.

RFPs and RFQs: Legality and Ethics

March 28th, 2010

Recently, I attended a webinar entitled "The ABCs of RFPs and RFQs".  This is one of those things that in my line of work (a manager of a travel model development group), I face occasionally.  I'm not an expert.  When presented the opportunity to get some guidance from some "experts" for free, I jumped on the chance.

I was disappointed.

Three things stuck out in my mind as being flat-out wrong.  The first was "The best case scenario is when you (the consultant) write the scope for the RFQ".  The second was "The best way is sole source contracts".  The third was constantly using RFP as a tool to limit the responses from consultants to only those that you want to respond.

Consultants Writing Scopes for RFQs

Looking at the AICP Code of Ethics, it seems that if a consultant writes the scope for the RFQ (or RFP), I feel it is in violation of Part A, 2a and 2c.  If a consultant is writing the scope for me, where is my professional judgement?  Does that judgement not extend to what I feel my needs (and my organizations needs) are?  Both of those are brought up in 2a.  Looking at 2c, which is avoiding a conflict of interest, it seems to me that if a consultant writes the scope for an RFQ, that is a direct conflict of interest - the consultant is going to write the scope that gives them an advantage (whether intentionally or unintentionally).

Sole Source?

When being audited by the State of Ohio Auditors, you are under extreme scrutiny when trying to sole-source a contract.  The reasons why are obvious.  A few years ago, my department attempted to sole-source a contract because it was a $30k contract and it seemed that there was only one firm that could do the job for that price.  While that may have been correct, there was several firms willing to try.  The job ultimately went to a firm that was NOT going to be the one that would have received the sole-source contract (there is a lot of talk that they may have taken a loss on the job, but I would venture a guess that the others would have as well).  Had the sole-source been allowed to continue, it would have been considered illegal under Ohio law and my organization would have been fined.

I can't type all this without bringing up another big issue that CAN negate the above.  General Planning Consultants and General Engineering Consultants.  The GPC and GEC contracts are always put up for RFQ, and handing a scope to a GEC or GPC consultant is NOT the same as sole-source.  This method is perfectly legal (it is open to public review and open to all consultants to submit statements of qualifications) and is a great way to get smaller (less than $100k, perhaps) jobs to consultants without them spending a lot of money trying to get smaller jobs.  They have to spend their marketing money up-front, but over the 3-5 year span, they have plenty of opportunity to make it back on smaller jobs that have very small marketing requirements.

RFPs Only to Certain Consultants?

Again, 2c, conflict of interest - public agencies cannot perform the work of the public good using the fewest tax dollars without having an open bid process.  Also, it is pretty likely that every state requires RFPs and RFQs to be advertised.  That being said, what's the point?  You're going to send the RFP to 2 or 3 consultants but post it on your website (and for us, the newspaper, state DOT website, and various other locations as required by law and our policy) for all to see?  Sounds like a pretty ineffective way to only target a few consultants.

If you only want certain consultants to respond, find a way to do it, legally, without giving the opportunity for other consultants to not compete for it.

Separating Intent and Unintended Effects

March 21st, 2010

On March 7, 2010 at Atlanta Motor Speedway (AMS), an interesting crash happened in the larger context of NASCAR.  Carl Edwards intentionally got into the side of Brad Kesolowski, causing Kesolowski to spin around, become airborne, and land on his side with momentum sending Kesolowski's car into the wall (video).  This was almost inverse of the Talledega spring race in 2009 where Edwards unintentionally came down on Kesoloski, spun around, became airborne, got hit by another car in the process, and hit the safety fence that separates the track from the stands(video).

The big difference between these two scenarios was intention.  Earlier in the race at AMS, Kesolowski got into the side of Edwards, causing Edwards a long repair and a poor finish.

NASCAR handed down a three-race probation to Edwards after parking him for the remainder of the race at AMS.  The debate as to whether that was the most appropriate disciplinary action have been swirling around NASCAR for weeks (and still is at the time of writing).

This post is not about whether NASCAR made the right or wrong decision, but rather how it relates to management.

You have to understand the history behind the wing.  If you've watched the videos above, you've seen two of three.  The other piece of history is at this video.  The scenario at AMS is the third time that a car has become airborne after being turned around.

The probation that Edwards faces (and no suspension or fine, mind you) was because Edward's intent was to mess up Kesolowski's 6th place finish with a spin to the infield.  Edwards didn't intend for the vehicle to flip, and the vehicle should not have flipped.  In fact, the severe crash was likely caused more by the wing on the back of the car (which has now been replaced with a spoiler), not by Edwards's intentionally spinning Kesolowski.

This is quite a conundrum for NASCAR.  They control the design of the car very strictly.  They also said that the drivers could use a little less restraint after feeling a lot of criticism over the 2009 season where they made rules that limited the drivers actions.  Drivers and teams are not allowed to make decisions as to whether they use the wing or not.  They have to use it.

The important thing here is, as a manager, make the decision looking at all pieces of information and all parts of history.  Look at what you've told your employees.  Look at what has happened in the past that your employees should have been aware of.  Look at what you would have done in that situation, particularly if you weren't a manager.  Discuss the issue with the employees involved.  Do not make rash decisions and do not let emotions be the only thing that guides your decisions.

Romanian street sign warns drivers of 'drunk pedestrians' - Telegraph

March 15th, 2010

In what is perhaps an accidental approach to reducing pedestrian crashes using the first step of "the three Es" (education, enforcement, engineering), Pecica, Romania has installed signs that warn of drunk pedestrians ahead.

While a little odd, I applaud the mayor for experimenting with a low-cost, low-impact way to handle the problem.  I hope it works.

Romanian street sign warns drivers of 'drunk pedestrians' - Telegraph.

Former DOT Secretary weighs in on Transportation Bill

October 15th, 2009

Reference: National Journal Online -- Insider Interviews -- Bush DOT Chief Discusses Reauthorization.

I agree with the thoughts of increased tolling and more fees other than the gas tax.  I also agree with $1B per year for technology, but it has to be managed right.

I'm also glad that the performance measures are measurable:

  • Congestion (we can measure that - it is the percent of a region's network that is operating with a demand greater than its capacity)
  • Costs (we can measure that, although we have to watch how we do it, as we don't want to have a system be considered bad if gas prices hit $4/gallon)
  • Safety (we DO measure this - it is the number of injuries and deaths on the road)

What are those little green boxes???

April 11th, 2009

It is the start of traffic counting season in Ohio. Each year, we get about 7 months to count the cars on the road. With my involvement in this type of work, I hear a lot of horror stories. First off, I wanted to discuss how these things work and how the data is used and cannot be used, and then show some of the war stories.

Traffic Counter on side of road

Traffic Counter on side of road

First off: how these things work

Those that have been around for 30 or more years may remember when some gas stations had a hose that rang a bell to call a station attendant to pump your fuel. Those that don't should watch Back to the Future. This is the same basic concept for most traffic counters. There are hoses that go across the road, and based on what the sensors feel and the time between them, these little green (or sometimes gray) boxes calculate the number of axles, distance between them (which can be used to derive the type of vehicle), and the speed.

I know that speed is a big issue with a lot of people. After all, some of these counters are being used for speed studies to see if they want to put a cop on a road at a certain time. This does happen, despite my wishes that cops and others would use less-detectable methods for enforcement. There are two other ways that counts, with speed, can be taken. One is by RADAR (the same thing they use for active speed enforcement). Mind you, for speed sampling, RADAR is pretty useful when installed correctly, and the boxes can be hidden quite well. The other is using magnetic loops. There are portable models of these that sit in the lane and are difficult to see (and also susceptible to theft). There are also permanent models that can be completely hidden from view.

One thing I can say with ALL hose counters: WE CANNOT USE THEM FOR SPEED ENFORCEMENT! The units do not have any cameras (etc), so if you speed while going over them, we know you did, but we don't know who you are!

Second off: How We Use The Data We Get From These Things

This one differs by jurisdiction, but most use it for traffic studies. Speed, count, and vehicle type are very useful for roadway improvement design. Another use is for travel model validation. We (specifically me, since it is my job) use this to ensure that the region's travel model is accurate so when we use it to plan billions of dollars in improvements, we know we're not just guessing, which would be a waste of money.

Law enforcement will use the number of speeders per unit of time to plan when to run patrols. As I indicated, I wish they wouldn't use hose counters for this, but they do, and the data they get is accurate. However, hoses are pretty conspicuous, which is why I wish they wouldn't use them.

We cannot use the data in court. You cannot be detected to be going 45 MPH in a 25 MPH zone based on a traffic counter. The counters do not have cameras in them, and none that I know of can connect to a camera. A camera would be required to prove who was speeding. Without the connection, it would be difficult to prove, since the times would have to be the same, the counter has to be operating perfectly, and the hoses have to be measured very precisely. Some states also forbid the use of cameras for passive law enforcement (a cop can actively use a RADAR+camera, but not mount one on a pole and get every car that is speeding).

The War Stories

I have two, both given to me by a salesperson for Jamar Tech, one of the leading traffic counter manufacturers.

City of Boston Thinks a Counter is a Bomb. This one is proof that some cops don't use hose counters, else they would have known what this unit is.

Counter burned, likely by an accelerant. PDF from Jamar, which the salesperson sent me just after I bought 8 counters from him.

Don't Mess With Them!

It amazes me that 1 month into the season, I've had to replace several hoses because of cut or stolen hoses. This is your tax dollars at work. The more hoses we have to replace, the less money we have to improve the roads.

Travel Demand Modeling 101 Part 1: Terminology

August 22nd, 2008

It occurred to me that many people likely do not understand all of the terminology of travel demand models.  Because of this, I felt the need to list many of them here. Read the rest of this post... »

Random Thought: Road Nicknames

June 4th, 2008

I've occasionally seen some road nicknames that are particularly good.  A few that I've heard:

  • Malfunction Junction (I-275 and I-4, Tampa, FL)
  • The Riddle in the Middle (Alaska Way, Seattle, WA)
  • Spaghetti Junction (I-85 and I-285, Atlanta, GA)

I've also started calling a strech of Columbia Parkway (Cincinnati, OH) "The Suicide Side", which is a 45 MPH arterial that everyone goes 60 MPH.  The divider is a double-yellow line... only.

Got any more?  Add 'em in the comments.

Four Step Model Explained: Trip Generation

June 3rd, 2008

Trip generation is likely one of the easiest parts of the four step process.  Normally, the most difficult part of dealing with trip generation is getting the input socioeconomic (population and employment) data correct.  This post explains how trip generation is calculated in the model... Read the rest of this post... »

Introduction to the Four Step Travel Demand Model

May 27th, 2008

The center of most travel demand models is the "Four Step Model".  This model was created in the 1950s to determine the demand on roadways.  The four steps include:

  1. Trip Generation
  2. Trip Distribution
  3. Mode Choice
  4. Trip Assignment

Read the rest of this post... »