UASI grant decisions: garbage in, garbage out?
Yesterday the Department of Homeland Security announced the FY 2006 state & local homeland security grant allocations, an announcement that provoked an immediate firestorm of anger in New York City and Washington, DC, due to the drastic funding cuts to the two cities that were attacked on 9/11. Homeland Security Committee chairman Peter King (R-NY) said that DHS had "declared war on New York," and today noted (acc. to CQ) that his conversation with DHS Under Secretary George Foresman was "the most heated conversation I’ve ever had with an appointed official" and that he "smelled incompetency" and DHS middle-managers were "phonies." An editorial in the New York Daily News called for Sec. Chertoff to be fired.
That firestorm continued to grow today, coming to a boil at an event at the Brookings Institution on homeland security at which Sec. Chertoff was speaking this afternoon. I attended the event, and a large share of his prepared remarks were a defense of these grant allocation decisions. His key argument was that last year's funding of $207 million for NYC was a "make-up year" for 2004, when NYC received only $47 million - and the new funding level of $125 million was not a cut, but a regression to the mean.
But Chertoff was called to task by a reporter to explain the one-page memorandum that was sent to New York explaining its funding cut - a memo that states that there are zero "national monuments and icons" in New York City - a pronouncement that has raised the ire of New Yorkers (but also inspired some humor). Chertoff tried to rationalize the decision, stating that key icons like the Statue of Liberty and Empire State Building were counted in different categories, but his defense on this point was unconvincing. At the very least, it's a management failure that this memo was sent to NYC officials with a statistic that is so glaringly off-base at face value.
I think this decision on NYC could be another example of the garbage-in, garbage-out (GIGO) problem, which I argued in April was a factor in the decision to cut Las Vegas from the list of high-risk cities. Some of the metrics on the one-page fact memo seem flawed in one way or another; for example, using the "quantity" of various asset types in these calculations fails to quantify their symbolic values, and using the quantity of threat reports from cities fails to account for different levels of discernment across cities about what constitutes suspicious activity.
I don't think that these decisions were political; if you believe this, then you have to explain why Dallas-Fort Worth's funding was cut drastically and North New Jersey's funding skyrocketed. And I think it's a wise move on the part of DHS to require states and cities to develop detailed spending plans prior to funding, which could have been the cause of NYC's funding cut; according to news reports, NYC's funding was slashed in part because of the salary & overtime requests in their proposal. States and cities who were shortchanged are certainly likely to take this process more seriously next year - which can only improve the effectiveness with which these funds are spent.
But I also think that DHS's processes and methodologies for assessing risk are still immature, and they have become over-fixated on data (cf. Sec. Chertoff's Mother of All Spreadsheets) of questionable value, and insufficiently focused on the holistic, qualitative elements of threat assessment. You can run the numbers in any which way, but that shouldn't change the fact that NYC and DC are the top two terrorist targets in the United States - by far - and deserve to be treated as exceptional cases when these funding decisions are made.
Update (6/2): Sec. Chertoff's remarks at Brookings are now available here. The criticism of DHS continued in Friday's papers, with one story in the Washington Post noting that Washington, DC was in the bottom tier of risk for the state grant program (which is different from the Urban Area Security Initiative, where the DC region was ranked highly), and another Post story criticizing DHS Asst. Secretary Tracy Henke for favoring her home state of Missouri in the allocation process, and the New York Daily News skewering DHS with the headline "Chertoff His Rocker". Some of these criticisms might be unfair or out-of-context, but until DHS opens the kimono on the grant decision-making process, it looks like "garbage-in, garbage-out."
That firestorm continued to grow today, coming to a boil at an event at the Brookings Institution on homeland security at which Sec. Chertoff was speaking this afternoon. I attended the event, and a large share of his prepared remarks were a defense of these grant allocation decisions. His key argument was that last year's funding of $207 million for NYC was a "make-up year" for 2004, when NYC received only $47 million - and the new funding level of $125 million was not a cut, but a regression to the mean.
But Chertoff was called to task by a reporter to explain the one-page memorandum that was sent to New York explaining its funding cut - a memo that states that there are zero "national monuments and icons" in New York City - a pronouncement that has raised the ire of New Yorkers (but also inspired some humor). Chertoff tried to rationalize the decision, stating that key icons like the Statue of Liberty and Empire State Building were counted in different categories, but his defense on this point was unconvincing. At the very least, it's a management failure that this memo was sent to NYC officials with a statistic that is so glaringly off-base at face value.
I think this decision on NYC could be another example of the garbage-in, garbage-out (GIGO) problem, which I argued in April was a factor in the decision to cut Las Vegas from the list of high-risk cities. Some of the metrics on the one-page fact memo seem flawed in one way or another; for example, using the "quantity" of various asset types in these calculations fails to quantify their symbolic values, and using the quantity of threat reports from cities fails to account for different levels of discernment across cities about what constitutes suspicious activity.
I don't think that these decisions were political; if you believe this, then you have to explain why Dallas-Fort Worth's funding was cut drastically and North New Jersey's funding skyrocketed. And I think it's a wise move on the part of DHS to require states and cities to develop detailed spending plans prior to funding, which could have been the cause of NYC's funding cut; according to news reports, NYC's funding was slashed in part because of the salary & overtime requests in their proposal. States and cities who were shortchanged are certainly likely to take this process more seriously next year - which can only improve the effectiveness with which these funds are spent.
But I also think that DHS's processes and methodologies for assessing risk are still immature, and they have become over-fixated on data (cf. Sec. Chertoff's Mother of All Spreadsheets) of questionable value, and insufficiently focused on the holistic, qualitative elements of threat assessment. You can run the numbers in any which way, but that shouldn't change the fact that NYC and DC are the top two terrorist targets in the United States - by far - and deserve to be treated as exceptional cases when these funding decisions are made.
Update (6/2): Sec. Chertoff's remarks at Brookings are now available here. The criticism of DHS continued in Friday's papers, with one story in the Washington Post noting that Washington, DC was in the bottom tier of risk for the state grant program (which is different from the Urban Area Security Initiative, where the DC region was ranked highly), and another Post story criticizing DHS Asst. Secretary Tracy Henke for favoring her home state of Missouri in the allocation process, and the New York Daily News skewering DHS with the headline "Chertoff His Rocker". Some of these criticisms might be unfair or out-of-context, but until DHS opens the kimono on the grant decision-making process, it looks like "garbage-in, garbage-out." 