Matt Hampel | matth.org

Original wiki page (you should use the version there) — copied here with a more descriptive title with the hope that it becomes easier to find.

#Make a new directory called "tmp" and a sub directory called "merged"
mkdir tmp
mkdir tmp/merged

#copy all zipped files to the "tmp" directory and then "cd" into it
cp *.zip tmp
cd tmp

#unzip all the .zip archives
find . -name "*.zip" -exec unzip '{}' \;

#delete all .zip archives
rm *.zip

# move a single shapefile (and the cooresponded .shx, .dbf, etc files) to the
# "merged" directory (exchange 'myshape*' for the name of one of your shapefiles
# keeping the '*' at the end of the name)
find . -name 'myshape*' -exec mv '{}' merged \;

#Batch merge all the remaining shapefiles from the tmp dir into the copied
# file in the merge dir (exchange 'myshape' for the name of the copied shapefile)
for i in $(ls *.shp); do ogr2ogr -f 'ESRI Shapefile' -update -append merged $i -nln myshape
done

You can’t do it, but you can use SEMI JOINs.

From the examples:

  SELECT a.key, a.value
  FROM a
  WHERE a.key in
   (SELECT b.key
    FROM B);

can be rewritten to:

   SELECT a.key, a.val
   FROM a LEFT SEMI JOIN b on (a.key = b.key)

(Mostly a note-to-self)

At far zoom levels:

Use neighborhood names instead of major street names

Just outline highways

thin light lines for minor streets

As you get closer, use the same color for minor streets, but now outline them (white/lighter center)

Put highways below other road layers

This round from looking at the devseed Baltimore local maps

Basic Graham Cracker Crust:

• 1/4 c. butter (melt in Microwave)
• 1/4 c. sugar
• 1 tsp. cinnamon
• 1 c. graham cracker crumbs

Mix and press into a 9 inch pie pan. (Use springform pan.).

Cheesecake:

• 2 (8 oz.) cream cheese, softened (in Microwave)
• 3 eggs
• juice from 1 lemon
• ½ cup sugar (don’t remember, maybe a bit more, just try).

• 8 oz. sour cream
• 1tablespoon vanilla

Preheat oven to 350 degreesBeat cheese until fluffy; add eggs, sugar and  lemon juice. Pour into prepared pan; bake 50 to 55 minutes or until cake springs back when lightly touched. Cool for a few minutes;  Mix sour cream with vanilla  and  spread  oncheesecake. Bake at 400 for about 5-10 minutes.

Adapted from somewhere.

Datasets often come as Excel files with multiple sheets (for example, one per year). If you don’t have Excel, you can use OpenOffice and this macro to export all the sheets as separate CSV files. It takes a directory path, searches for all Excel files in that directory, and then saves each sheet in each file as CSV with the name (file name)_(sheet name).csv

To run it, you need to:

1. Open Office Calc, click Tools -> Macros -> Organize Macros -> OpenOffice.Org Basic.
2. Click the new button
3. Paste in the macro code
4. Change the file directory (cFolder variable, around like 29) to some folder. This is where OpenOffice will search for Excel files and save the results.
5. Run the macro (you might have to go through Tools -> Macros -> Run Macro)

There probably is a way to do this directly from the command line.

I am posting these documents with little background; updates are possible as I collect more files. These should be on DetroitWiki instead.

Doug Guthrie of the The Detroit News wrote an overview of the consent decrees from as of 2009.

Consent Judgement Use of Force and Arrest and Witness Detention (03-72258)

Consent Judgement Conditions of Confinement (03-72258)

Both to be signed by  plaintiffs:

RUTH C. CARTER Corporation Counsel City of Detroit
BRENDA E. BRACEFUL Deputy Corporation Counsel
KWAME M. KILPATRICK Mayor City of Detroit
JERRY A. OLIVER, SR. Chief of Police City of Detroit

June 12, 2003

I found these by searching for the exact file names on usdoj.gov

There also are several technical assistance letters from the Department of Justice from 2002, which are more readable than the court filings.

1.      Use of Force – March 6, 2002

2.      Holding Cell – April 4, 2002

3.      Arrest and Witness Detention – June 5, 2002

Sheryl Robinson Wood served as the independent monitor. The reports of the Independent Monitor of the Detroit Police Department from January 2004 through September 2009 are online.

Here is a PDF (55mb) of the 2009 Detroit Journal of City Council, the official record of the Council’s proceedings. It’s similar to minutes in that it lists all decisions, votes, and other recorded commentary. I’ll post the full story of how this PDF came to be here “soon”.

The Center for Educational Performance and Information (CEPI) at the State of Michigan publishes raw data about schools, students, staff, and finances. This includes enrollment, demographics, safety & security, balance sheet, revenue, expenditure, and a lot more.

After installing DocSplit, I got this error when attempting to extract text from a PDF via OCR:

$>docsplit text ./BSED.pdf --pages all
execvp failed, errno = 2 (No such file or directory)
gm convert: "gs" "-q" "-dBATCH" "-dMaxBitmap=50000000" "-dNOPAUSE" "-sDEVICE=ppmraw" "-dTextAlphaBits=4" "-dGraphicsAlphaBits=4" "-r200x200" "-dFirstPage=1" "-dLastPage=1" "-sOutputFile=/var/folders/D5/D5Cief2MHW8vTHOI8nkyPU+++TM/-Tmp-/d20101117-83113-1f9etpp/gmUoD1ux" "--" "/var/folders/D5/D5Cief2MHW8vTHOI8nkyPU+++TM/-Tmp-/d20101117-83113-1f9etpp/gmXsXajF" "-c" "quit".
gm convert: Postscript delegate failed (./BSED.pdf).


I am on Mac OS 10.6. The solution was to upgrade ghostscript to the latest version:

$>brew install ghostscript.

If you are interested in either, both, or related subjects, please do get in touch: matthew.hampel@gmail.com

This post is way too long in coming, especially because I was disappointed to find no other like it when I googled those keywords earlier this year.

Some starting points: (hopefully I’ll be able to add more)

http://detroitwiki.org/Data_about_Detroit

http://datadrivendetroit.org/ (501c3, I’m interning there this semester)