Shape Escape

Well it's been a while, so just a quick note: Since the last post, I started working full time for Google. And with that out of the way, here's a post on how and why I made shpescape.com, which lets you upload shapefiles to Google Fusion Tables.

Why shpescape?

Google Fusion tables makes it easy to import and visualize data from spreadsheets and KML, and while it has increasingly robust spatial support it does not currently let you upload shapefiles directly. And since shapefiles are still incredibly common in the wild, I thought I'd make a quick tool to let people upload shapefiles to Fusion Tables.

Which platform?

I thought I'd try Google App Engine to avoid any hosting costs (given this will likely not be an extremely popular website), but while there's a decent shapefile reader or two for python, there's not a lot of support for things like reprojecting and other geometry manipulation without additional c++ libraries that App Engine won't run. So I just went for a simple GeoDjango app.

Authentication

I used my colleague Kathryn's Fusion Tables python client to handle the authentication (OAuth). And I decided against having OpenID in the mix as well for actually associating an account with various uploads. The downside is that you can't log in and view your previous uploads. But you can always go to the main Fusion Tables page to see all your tables, and the upside was one less thing for me to consider (for example, if you are logged in with multiple accounts in the same browser, OAuth does not return which account gave permissions for). [Edit: It turns out you can actually request the email address of an authorized user using the scope noted at http://sites.google.com/site/oauthgoog/Home/emaildisplayscope]

Handling a Shapefile Upload

I used a simple fork of Dane Springmeyer's django-shapes app to handle the shapefile import. The customizations let users upload a zipfile that has a shapefile in a subfolder, and/or multiple shapefiles in a single zip. I had never really noticed shapefiles being zipped up this way, and it really surprised me how common these scenarios are with shapefiles from various US counties and other agencies -- my first 3 test users all had their uploads fail until I added this. After the upload is verified as valid, it creates a shapeUpload object, which is processed separately so the end user can view a reloading page with updated status.

Processing the Upload

My initial attempt was pretty straightforward:
  • Attempt to get the projection of the shapefile (from the .prj)
  • For each feature, get it's KML and attributes
  • Upload 'em to Fusion Tables, ensuring a max of a few hundred, or <1MB, at a time (the API can handle at most 500 rows and 1MB per POST)


Additional Features

Next up, I started adding a few extra bits, which led to an import method begging for a refactor.
  • Simplification for KML over 1M characters long (which is the max characters allowed by Fusion Tables per cell)
  • Process/Upload 10k rows at a time (so we don't use too much memory for very large shapefiles)
  • Added numeric styling columns for string fields that don't have too many unique values (Fusion Tables only allows robust styling like gradients and buckets on numeric fields)
  • Allow users to specify some additional geometry columns:
    • Simplified Geometry
    • Centroid (only works for polygon shapefiles)
    • Buffered Centroid (so you can apply the more robust polygon styling rules on the 'centroid')


Finishing up

This whole project was a pretty quick attempt at what I hope is a useful solution to a common problem, so any comments on how to make it better are appreciated. And if you want to see how it all works in more detail, I also open sourced the code. Enjoy!

Comments

Unknown said…
Hello. This is great. Though, it doesn't work well with non latin languages. Do you have any idea how to fix non latin characters?
priya sharma said…
Such A nice post... thanks For Sharing !!Great information for new guy like Happy New year 2020
anita said…
Such A nice post... thanks For Sharing !!Great information for new guy like Hanuman Chalisa Lyrics
archna said…
Good effort and This is a very valuable concept.Such A nice post... thanks For Sharing !!Great information for new guy like showbox for android
gadgetsdora said…
Your style is very unique compared to other folks I've read stuff from. Many thanks for posting when you have the opportunity, Guess I will just bookmark this site.
tech
centre99 said…
it is really a great and helpful piece of info. I am glad that you shared this helpful information with us. Please keep us informed like this. Thank you for sharing.
china embassy singapore
travel5212 said…
Amazing Article,Really useful information to all So, I hope you will share more information to be check and share here.thanks for sharing .
Website: Vietnam Cycling Tours

riya singh said…
I enjoyed over read your blog post. This was actually what i was looking for and i am glad to came here!
Website: artificial jewellery online
jenifer027 said…
MyAssignmentHelpNow provides assignment help service to the students of Australia...We are well known and most demanding writing service provider over there. If you are looking for the best writing service provider over there, then we are the one-stop-shop for you.

MyAssignmentHelp
anthonyhall said…
an interesting article, it was pleasant to read, well written, I myself sometimes write articles, and it helps me https://soclikes.com/
Banumadhu said…
Useful article which was very helpful. also interesting and contains good information.
to know about python training course , use the below link.

Python Training in chennai

Python Course in chennai
pkcartoon said…
Post is very good its amzazing post I love them thanks for sharing.
visit here - Desi Call Recordings in Hindi
rajmohan1140 said…
Always good to find good articles on the net. Thank you. This is very valuable thing shared, I thanked the writer.

Java Training in Chennai

Java Course in Chennai
it is really a great and helpful piece of info. I am glad that you shared this helpful information with us. Please keep us informed like this. Thank you for sharing.
Website: Online punjab state lottery
Ankit Ron said…
Very genuine post i love to share like this kind of article, eventually thanks for sharing. ball wala game
buyonline123 said…
Thanks for another great post. Where else could anyone get that kind of information in such a perfect way of writing? I've a presentation next week, and I'm on the look for such information.
buy electronics online
marketshare said…
I am new here. I like your post very much. It is very usefull post for me.
website: Global 5G Infrastructure Market
Anonymous said…
Great post and I got extremely information from your article. I appreciate your work and requesting you to please keep posting other articles.
Speedify Crack
Crack Software said…
I’m impressed, I must say. Genuinely rarely can i encounter a blog that’s both educative and entertaining, and without a doubt, you have hit the nail on the head. Your concept is outstanding; the issue is something that too little individuals are speaking intelligently about

Crack Software

PC License Key

World PC Crack
Gaurish said…
I always like to read a quality content having accurate information regarding the subject and the same thing I found in this post.

Website : Website Development Company |
Gaurish said…
Great information, i was searching of this kind of information, thank you very much for sharing with us.

Website : Craigslist Posting Service for Car Dealers |
Gaurish said…
This is really amazing website that I have been found on google regarding website Blog Commenting sites. and I would like to thank admin who also given us to post the link on his side.

Website : Lubbock moving company |
Gaurish said…
This was something I was looking for, really helpful, and great work is done. Thank you so much for sharing such valuable information.

Website : Car Auction Software |
Gaurish said…
It’s really a cool and helpful piece of information. I am glad that you shared this useful information with us. Please keep us up to date like this. Thanks for sharing.

Website : Best CRM for Small Businesses |
Gaurish said…
I am really like it very much for the interesting info in this blog that to this website is providing the wonderful info in this blog that to utilize the great technology in this blog.

Website : Web Development Company in Gwalior |
sattamatka25 said…

Thank you so much for sharing this article. I really love it. This is a very nice one and gives in-depth information. I am really happy with the quality and presentation of the article. I’d really like to appreciate the efforts you get with writing this post. Thanks for sharing.
website: Matka Result
Evo Syah said…
Evo Syah adalah seorang Freelancer Website Developer Bali, dia sangat menyukai hobbynya dalam menelusuri setiap blok yang Anda di internet. Dia memiliki banyak pengetahuan tentang Digital Marketing dan bagaimana penerapan dalam bisnis.
Evo Syah said…
Satya Puri Studio is Bali architects have worked on historically-inspired homes & villa around the world — including “Lux Villa in Bali; Sukh Sagar Haveli in Jodhpur, Rajasthan; Kahani Paradise in Gokarna. Their work shows inspiration across history and styles, tied together with the goal of enhancing life through good design. Above all, Satya Puri Studio Architects aims to produce personalized living environments.
Evo Syah said…
SEO adalah proses meningkatkan visibilitas dan peringkat website bisnis Anda di laman hasil penelusuran mesin telusur, seperti Google, Bing, dan Yahoo, sehingga memudahkan pelanggan menemukan produk atau layanan Anda secara online.

Kami menawarkan layanan Jasa SEO terbaik di Bali dengan Strategi Optimasi Situs Website secara tepat dan akurat sesuai dengan prinsip kerja mesin pencari.
Evo Syah said…
When you think Indonesia, you probably picture its Labuan Bajo, and sunny beaches in Canggu. Yet, many people don’t realize that Indonesia is also a great place to start a business. According to International Labour Organization, there are about 700 thousand small businesses in Indonesia in 2018, which 57 million are small and medium sized enterprises Start Business in Indonesia

But what makes Indonesia so great? Major cities like Jakarta, Bandung, Yogyakarta, and Surabaya, an impressive university system, and low taxes means the Sunshine country is ripe with opportunity for savvy business owners. Keep reading to learn more about why Indonesia could be the perfect spot to start your new business!
crack software said…
Remarkable! Its in fact remarkable paragraph, I have got much clear idea concerning from this article.

explorecrack.com

brave-browser-crack

bandicam-crack-product-key

wondershare-filmora-crack
Marty Sockolov said…
CBD isolates. CBD (cannabidiol) is an active ingredient found in th CBD Cannabis plant that does not cause psychoactive "high". Instead, the active ingredient provides soothing energy, such as a stronger Ginkgo Biloba tea
Rani tyagi said…
I just would like to give a huge thumbs up for the great info you have here on this post. Thanks for sharing with us. Are you looking for Escort Services in Ahmedabad and Call Girls in Ahmedabad
Click For More Information Hire Call Girls Ahmedabad
Including fast withdrawals With a system that has high precision, meets international standards,can start playing for free. You will find the rules and the hand rank. Find other poker games that help hone your skills withPlay Money Texas Hold'em, Omaha and or join real money games. There is no better place to learn and play poker from us.

Marty Sockolov said…
it is not the most popular on this list, but fans do love its benefits ดูบอลฟรี . One of these is that no sign up is required, but you can sign up for free to get some convenient options. For instance, it provides intelligent content recommendations and a calendar for you to set up reminders. That way, you never miss your favorite live content. If you do miss it, though, you can always pull it up on Duball24hrs.com recordings.

unknown said…
Download Apk and Free Mod Apk For Android


https://apkzamn.com/
Maradona Jons said…
เราคือผู้นำด้านเกมพนันออนไลน์ Major168 เราคือผู้ให้บริการ คาสิโนออนไลน์ ที่ได้รับรองว่าดีที่สุดในประเทศไทย มีค่ายเกมส์ให้เล่นมากมาย Sagaming, Sexy bacarat, Dreamgame, Ebet, Wm casino, Vivo gaming ไม่ผ่านเอเย่นต์ ระบบปลอดภัยมีทีมงานดูแลตลอด 24ชม.

SAGAME88 แหล่งรวมเกมส์พนันออนไลน์ คาสิโนสด บาคาร่า กำถั่ว คาสิโนออนไลน์ ไฮโล รูเล็ต รับเครดิตฟรีเล่นได้ทุกเกมส์ โบนัสสมาชิกใหม่เพียบ พร้อมระบบฝากถอนออโต้ 10วิ เรามีทุกค่ายเกมส์ให้คุณเลือกเดิมพัน SA Game Sexy bacarat Dreamgame WM Casino VIVO Gaming Ebet เล่นได้ทุกเกมส์

Our website ufabet provides betting services in the system of Auto Deposit-Withdrawal. Our members are not only in Thailand. Online football betting UEFA Bet market And this makes a guarantee that Ufabet168 is another reliable football betting website

ufa through the website UFABET1688 a web gambling online , one that integrated all the bets from online casinos , online casino , online , and also have a game a lot more to be chosen to play such games.
Along with it also has a live casino 24 hours a day. สมัคร ufa Bet online gambling website is available to play with all casinos, baccarat, slots, roulette, dice and many other card games. We have gathered it here. With many promotions Apply for UEFA Bet now, we are happy to serve you all. With a professional team ready to solve problems And serve all of you There is a team to support the service. And answer your questions 24 hours a day.
Scottishkilt said…
Great article! This is the type of information that are meant to
be shared across the internet. Thank you for sharing such a useful post. Very Interesting Post! I regularly follow this kind of Blog.
visit this page
Cracked Apps said…



I’m amazed, I have to admit. Rarely do I come across a blog that’s both equally educative and entertaining, and let me tell you, you have hit the nail on the head.Thanks a lot for sharing.
Freemake Video Converter Crack
Spotify Crack
EaseUS Todo Backup Crack
Smart Game Booster Crack
Malwarebytes Crack
Guitar Pro Crack
Recover My Files Crack
Device Doctor Pro Crack
thomas said…
Here's an example: Let's say you use your CareCredit card to pay for a $3,000 dental bill. You qualified for a 24-month 0% annual percentage rate promotional period. You do your best to pay it off, but there's still $1,000 left after two years.

fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
thomas said…
Best Airline Credit Cards: U.S. News’ overall issuer satisfaction rating, ease and flexibility of rewards redemption, airline rewards earning rate, rewards redemption value, annual fee, APR, sign-up bonus value, cardholder benefits, foreign transaction fee, and balance transfer fee.

inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
thomas said…
"Everything about the way we operated (this year) was a little bit different," Weston says. "I would say to my clients: Anytime you are anticipating or coming into a major life change, call me. Say, 'Hey Cari, we're moving, we want to sell a house, we want to buy a new car.' Those are the kinds of things that your client only has every five to 10 years. But then, think of how many of those things people were going through last year."

juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
thomas said…
Joshua Zimmelman, president of Westwood Tax & Consulting LLC in Rockville Centre, New York, says, "Employees have limits to what expenses they can deduct, but if you're an independent contractor or freelancer, you can deduct many business expenses. For a Lyft or Uber driver, for example, this might include gas, parking fees, car maintenance and repairs, auto insurance and any other expenses you incur for your car."

localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
sam said…
Thanks for Sharing this Valuable Information with us: this is very useful for me. Keep it Up. Trending Blogs India 2021, Most Popular Blog Topics India 2021, Trending Topics in India 2021
Visit here - breaking news india pakistan

Popular posts from this blog

TopoJSON

Featureserver on AppEngine