Shape Escape

Well it's been a while, so just a quick note: Since the last post, I started working full time for Google. And with that out of the way, here's a post on how and why I made shpescape.com, which lets you upload shapefiles to Google Fusion Tables.

Why shpescape?

Google Fusion tables makes it easy to import and visualize data from spreadsheets and KML, and while it has increasingly robust spatial support it does not currently let you upload shapefiles directly. And since shapefiles are still incredibly common in the wild, I thought I'd make a quick tool to let people upload shapefiles to Fusion Tables.

Which platform?

I thought I'd try Google App Engine to avoid any hosting costs (given this will likely not be an extremely popular website), but while there's a decent shapefile reader or two for python, there's not a lot of support for things like reprojecting and other geometry manipulation without additional c++ libraries that App Engine won't run. So I just went for a simple GeoDjango app.

Authentication

I used my colleague Kathryn's Fusion Tables python client to handle the authentication (OAuth). And I decided against having OpenID in the mix as well for actually associating an account with various uploads. The downside is that you can't log in and view your previous uploads. But you can always go to the main Fusion Tables page to see all your tables, and the upside was one less thing for me to consider (for example, if you are logged in with multiple accounts in the same browser, OAuth does not return which account gave permissions for). [Edit: It turns out you can actually request the email address of an authorized user using the scope noted at http://sites.google.com/site/oauthgoog/Home/emaildisplayscope]

Handling a Shapefile Upload

I used a simple fork of Dane Springmeyer's django-shapes app to handle the shapefile import. The customizations let users upload a zipfile that has a shapefile in a subfolder, and/or multiple shapefiles in a single zip. I had never really noticed shapefiles being zipped up this way, and it really surprised me how common these scenarios are with shapefiles from various US counties and other agencies -- my first 3 test users all had their uploads fail until I added this. After the upload is verified as valid, it creates a shapeUpload object, which is processed separately so the end user can view a reloading page with updated status.

Processing the Upload

My initial attempt was pretty straightforward:
  • Attempt to get the projection of the shapefile (from the .prj)
  • For each feature, get it's KML and attributes
  • Upload 'em to Fusion Tables, ensuring a max of a few hundred, or <1MB, at a time (the API can handle at most 500 rows and 1MB per POST)


Additional Features

Next up, I started adding a few extra bits, which led to an import method begging for a refactor.
  • Simplification for KML over 1M characters long (which is the max characters allowed by Fusion Tables per cell)
  • Process/Upload 10k rows at a time (so we don't use too much memory for very large shapefiles)
  • Added numeric styling columns for string fields that don't have too many unique values (Fusion Tables only allows robust styling like gradients and buckets on numeric fields)
  • Allow users to specify some additional geometry columns:
    • Simplified Geometry
    • Centroid (only works for polygon shapefiles)
    • Buffered Centroid (so you can apply the more robust polygon styling rules on the 'centroid')


Finishing up

This whole project was a pretty quick attempt at what I hope is a useful solution to a common problem, so any comments on how to make it better are appreciated. And if you want to see how it all works in more detail, I also open sourced the code. Enjoy!

Comments

Unknown said…
Hello. This is great. Though, it doesn't work well with non latin languages. Do you have any idea how to fix non latin characters?
priya sharma said…
Such A nice post... thanks For Sharing !!Great information for new guy like Happy New year 2020
archna said…
Good effort and This is a very valuable concept.Such A nice post... thanks For Sharing !!Great information for new guy like showbox for android
gadgetsdora said…
Your style is very unique compared to other folks I've read stuff from. Many thanks for posting when you have the opportunity, Guess I will just bookmark this site.
tech
centre99 said…
it is really a great and helpful piece of info. I am glad that you shared this helpful information with us. Please keep us informed like this. Thank you for sharing.
china embassy singapore
travel5212 said…
Amazing Article,Really useful information to all So, I hope you will share more information to be check and share here.thanks for sharing .
Website: Vietnam Cycling Tours

jenifer027 said…
MyAssignmentHelpNow provides assignment help service to the students of Australia...We are well known and most demanding writing service provider over there. If you are looking for the best writing service provider over there, then we are the one-stop-shop for you.

MyAssignmentHelp
pkcartoon said…
Post is very good its amzazing post I love them thanks for sharing.
visit here - Desi Call Recordings in Hindi
rajmohan1140 said…
Always good to find good articles on the net. Thank you. This is very valuable thing shared, I thanked the writer.

Java Training in Chennai

Java Course in Chennai
Ankit Ron said…
Very genuine post i love to share like this kind of article, eventually thanks for sharing. ball wala game
buyonline123 said…
Thanks for another great post. Where else could anyone get that kind of information in such a perfect way of writing? I've a presentation next week, and I'm on the look for such information.
buy electronics online
marketshare said…
I am new here. I like your post very much. It is very usefull post for me.
website: Global 5G Infrastructure Market
Anonymous said…
Great post and I got extremely information from your article. I appreciate your work and requesting you to please keep posting other articles.
Speedify Crack
Crack Software said…
I’m impressed, I must say. Genuinely rarely can i encounter a blog that’s both educative and entertaining, and without a doubt, you have hit the nail on the head. Your concept is outstanding; the issue is something that too little individuals are speaking intelligently about

Crack Software

PC License Key

World PC Crack
Gaurish said…
I always like to read a quality content having accurate information regarding the subject and the same thing I found in this post.

Website : Website Development Company |
Gaurish said…
This was something I was looking for, really helpful, and great work is done. Thank you so much for sharing such valuable information.

Website : Car Auction Software |
Gaurish said…
It’s really a cool and helpful piece of information. I am glad that you shared this useful information with us. Please keep us up to date like this. Thanks for sharing.

Website : Best CRM for Small Businesses |
Evo Syah said…
Evo Syah adalah seorang Freelancer Website Developer Bali, dia sangat menyukai hobbynya dalam menelusuri setiap blok yang Anda di internet. Dia memiliki banyak pengetahuan tentang Digital Marketing dan bagaimana penerapan dalam bisnis.
Evo Syah said…
SEO adalah proses meningkatkan visibilitas dan peringkat website bisnis Anda di laman hasil penelusuran mesin telusur, seperti Google, Bing, dan Yahoo, sehingga memudahkan pelanggan menemukan produk atau layanan Anda secara online.

Kami menawarkan layanan Jasa SEO terbaik di Bali dengan Strategi Optimasi Situs Website secara tepat dan akurat sesuai dengan prinsip kerja mesin pencari.
Evo Syah said…
When you think Indonesia, you probably picture its Labuan Bajo, and sunny beaches in Canggu. Yet, many people don’t realize that Indonesia is also a great place to start a business. According to International Labour Organization, there are about 700 thousand small businesses in Indonesia in 2018, which 57 million are small and medium sized enterprises Start Business in Indonesia

But what makes Indonesia so great? Major cities like Jakarta, Bandung, Yogyakarta, and Surabaya, an impressive university system, and low taxes means the Sunshine country is ripe with opportunity for savvy business owners. Keep reading to learn more about why Indonesia could be the perfect spot to start your new business!
crack software said…
Remarkable! Its in fact remarkable paragraph, I have got much clear idea concerning from this article.

explorecrack.com

brave-browser-crack

bandicam-crack-product-key

wondershare-filmora-crack
Marty Sockolov said…
CBD isolates. CBD (cannabidiol) is an active ingredient found in th CBD Cannabis plant that does not cause psychoactive "high". Instead, the active ingredient provides soothing energy, such as a stronger Ginkgo Biloba tea
Including fast withdrawals With a system that has high precision, meets international standards,can start playing for free. You will find the rules and the hand rank. Find other poker games that help hone your skills withPlay Money Texas Hold'em, Omaha and or join real money games. There is no better place to learn and play poker from us.

Marty Sockolov said…
it is not the most popular on this list, but fans do love its benefits ดูบอลฟรี . One of these is that no sign up is required, but you can sign up for free to get some convenient options. For instance, it provides intelligent content recommendations and a calendar for you to set up reminders. That way, you never miss your favorite live content. If you do miss it, though, you can always pull it up on Duball24hrs.com recordings.

Along with it also has a live casino 24 hours a day. สมัคร ufa Bet online gambling website is available to play with all casinos, baccarat, slots, roulette, dice and many other card games. We have gathered it here. With many promotions Apply for UEFA Bet now, we are happy to serve you all. With a professional team ready to solve problems And serve all of you There is a team to support the service. And answer your questions 24 hours a day.
Scottishkilt said…
Great article! This is the type of information that are meant to
be shared across the internet. Thank you for sharing such a useful post. Very Interesting Post! I regularly follow this kind of Blog.
visit this page
Cracked Apps said…



I’m amazed, I have to admit. Rarely do I come across a blog that’s both equally educative and entertaining, and let me tell you, you have hit the nail on the head.Thanks a lot for sharing.
Freemake Video Converter Crack
Spotify Crack
EaseUS Todo Backup Crack
Smart Game Booster Crack
Malwarebytes Crack
Guitar Pro Crack
Recover My Files Crack
Device Doctor Pro Crack
thomas said…
Here's an example: Let's say you use your CareCredit card to pay for a $3,000 dental bill. You qualified for a 24-month 0% annual percentage rate promotional period. You do your best to pay it off, but there's still $1,000 left after two years.

fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
fishyfacts4u.com
thomas said…
Best Airline Credit Cards: U.S. News’ overall issuer satisfaction rating, ease and flexibility of rewards redemption, airline rewards earning rate, rewards redemption value, annual fee, APR, sign-up bonus value, cardholder benefits, foreign transaction fee, and balance transfer fee.

inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
inewshunter.com
thomas said…
"Everything about the way we operated (this year) was a little bit different," Weston says. "I would say to my clients: Anytime you are anticipating or coming into a major life change, call me. Say, 'Hey Cari, we're moving, we want to sell a house, we want to buy a new car.' Those are the kinds of things that your client only has every five to 10 years. But then, think of how many of those things people were going through last year."

juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
juicyfactor.com
thomas said…
Joshua Zimmelman, president of Westwood Tax & Consulting LLC in Rockville Centre, New York, says, "Employees have limits to what expenses they can deduct, but if you're an independent contractor or freelancer, you can deduct many business expenses. For a Lyft or Uber driver, for example, this might include gas, parking fees, car maintenance and repairs, auto insurance and any other expenses you incur for your car."

localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
localnewsbuzz.com
sam said…
Thanks for Sharing this Valuable Information with us: this is very useful for me. Keep it Up. Trending Blogs India 2021, Most Popular Blog Topics India 2021, Trending Topics in India 2021
Visit here - breaking news india pakistan
John P. Ketchum said…
There are puzzles on the app you solve using code, and there's also a blank coding editor to create your own apps and games. You can even submit your creation directly to the App Store using Swift Playgrounds. The application is 100% free and guides you step-by-step on how to create better apps using Swift.

Gadgets
Gaming
Health & Fitness
Lifestyle
Movie
Music
News
Nutrition
sam said…
Riverside Chevrolet is one of the best Used Car Dealers Peoria Il. Visit our website and check out our inventory of new and used vehicles. We are your one-stop shop for all your automobile needs.
Visit here - Used Car Dealers Peoria Il
Edison hope said…
Awesome! I thank you your input to this matter. It has been insightful. Thanks. Just keep posting like this.. Eligible travelers can obtain electronic Visa Turkey by filling an online form with their personal details and passport information.
Tom said…
Thank you for this work.. The UK citizens traveling to Azerbaijan can apply for online Azerbaijan visa for UK citizens. Get visa just in 3 steps apply online, upload document, and make payment.
essay writer said…
Your style is very unique compared to other folks I've read stuff from. Many thanks for posting when you have the opportunity, College online
essay writer said…
Thanks for Sharing this Valuable Information with us: this is very useful for me. Keep it Up
writing tutor
Edison hope said…
After reading this I thought it was extremely informative. I appreciate you taking the time and energy to put this information together. Eligible countries for South Africa visa. The South Africa e-Visa is an electronic visa that allows entry into South Africa. Once available, eligible citizens will be able to complete the online application process without having to visit an embassy or consulate.
Evo Syah said…
Apart from generic Furniture Bali designs that we mass manufacture and distribute globally, we do our best to experiment with furniture designs that we either get through our commercial or hospitality customers or we simply mix-match or tinker around with some of the well-known furniture designs till we end up with some striking furniture pieces . We developed expertise in working furniture made from both metal, bamboo & wood and we began our journey as a manufacturer of industrial furniture. That’s the point where our interest piqued and we relentlessly began pursuing the holy grail of furniture manufacturing – developing expertise in intricate furniture styles like vintage furniture, art deco furniture, villa furniture, restaurant furniture, hotel contemporary, Balinese furniture, and bamboo designs.
Robert smith said…
This is a topic that’s near to my heart… Many thanks! Where are your contact details though? E Visa for India is you can obtain online without the need of visiting the Indian Embassy and Consulate. Indian visa fees depend on your country and visa type.

deepaks42 said…
Hello there, I discovered your site by way of Google at the same time as looking for a
comparable matter, your site got here up, it seems to be good.
clean and clear face wash
jamesc32 said…
I am no expert, but I believe you just made an excellent point. As we all know how much Bill of Sale being used. This document is usually used for sale and purchase between two parties like Buyer & seller.
Faa bill of sale
manmeet1 said…
51 Lakhs distributed over prize money. Gandhi Brother’s lottery is Authorized and legal, assuring each player of a fair playing lottery.
nagaland lottery sambad

geetas32 said…
We, at Easy To Pitch focus on every industry space and our association with 500+ startups proves that. Once associated, we will ensure that your startup becomes pitch-perfect. Not only this we are a team that includes IIMians with various domain expertise.
startup pitch presentation
the study iQ said…
post... thanks For Sharing !!Great information for new guy like
Hanuman Chalisa Lyrics
Bhagwat Geeta in Hindi PDF
sarvanam
mounimeera said…
Thank you for giving good services https://caterersinhyderabad.in/"> Caterers in Hyderabad

delvers Allison said…
Wow.. Very informative article thanks for sharing please keep it up.. Pakistan visa fees depend on your visa type and nationality.

کرج آپارتمان فروشی، خرید و فروش خانه و آپارتمان در فردیس، کمال‌شهر، نظرآباد، محمدشهر، ماهدشت، مشکین‌دشت، چهارباغ، شهر جدید هشتگرد، اشتهارد، گرمدره، گلسار، کوهسار، تنکمان، طالقان، آسارا، خرید و فروش آپارتمان و خانه کلنگی و نوساز
فروش آپارتمان کرج
Maradona Jons said…
เล่นสล็อตออนไลน์ที่ดีที่สุดในโลก ทั้งหมดในที่เดียวโดยไม่ต้องดาวน์โหลดที่ BK8 – ค้นพบเกมสล็อตที่น่าตื่นเต้นและน่าดึงดูดที่สุด เสริมด้วยกลไกที่ยอดเยี่ยม ภาพที่สวยงาม และมีโอกาสชนะรางวัลแจ็คพอตสุดพิเศษ!สมัคร bk8thai
rom52 said…
Thank You for Providing Such insightful information. If someone is looking for the दिल जलाने वाली शायरी | shayari for true love | शायरी दमदार.
jameslpina said…
Tamiloneindia is a platform that allows gamers worldwide to sell and buy games-related items and services that are hard to locate, expensive or unsafe to purchase. We have the most up-to-date PC Game News, including tutorials and advice from gamers so that you can enhance your game. Check it out now!
Noah Ava said…

This is new knowledge for me, I am excited about it. thanks....India e tourist visa, You can get an online tourist visa to India and visit the beautiful religious/historical places of India etc.
Looking for finance assignment writing help, and it's becoming difficult to find one just try contacting assignment achievers top experts at your help at an affordable price.
https://www.assignmentachievers.com/service-detail/finance+assignment+help
https://www.assignmentachievers.com/service-detail/criminal+law+assignment+help
Maradona Jons said…
We will be adding more UK No Deposit Bonuses soon. Keep a look out as the latest UK no deposit casino bonuses appear here offering free spins with zero deposit and no deposit UK cash bonuses.http://E-Vegas.com
Maradona Jons said…
Dikenal sebagai bandar judi online terbaik tentunya Anda wajib tahu apa saja kelebihan kami. Dibawah ini kami akan menjabarkan satu persatu kelebihan kami. Slot deposit dana


Ronnie said…
Awesome dispatch! I am indeed getting apt to over this info, is truly neighborly my buddy. Likewise fantastic blog here among many of the costly info you acquire. Reserve up the beneficial process you are doing here. hanumanchalisalyrics
Sophia said…

Good morning sir, this is great. If you want to learn more about Turkish visas you can visit on our e visa Turkey website. And read here all info about Turkey evisa online.

Thank You for Providing Such insightful information. If someone is looking for the Quickbooks Support Phone Number in US.
Phuwadon said…
ฟีเจอร์การทำเงินผ่านทางเว็ปสล็อตออนไลน์ สล็อต JOKER อีกหนึ่งเว็ปพนันออนไลน์ทำเงินออนไลน์ ที่สามารถทำเงินได้อย่างมากที่สุดเท่าที่เคยมีมา เงินรางวัลโบนัสแตกง่าย โบนัสแจ็คพอตสูง.
Thank You for Providing Such insightful information. If someone is looking for the Quickbooks Customer Service Number in US.
สล็อตเว็บตรง นั้นเปรียยบได้เหมือนหัวเรือนำขบวนของเหล่าสล็อตเว็บตรงต่าง ๆ ที่มีในปัจจุบัน โดยทั้งหมดที่กล่าวมานั้น PG Slot เว็บตรงมักจะเป็นผู้นำเทรนอยู่เสมอและสิ่งที่นำเทรนนั้นคือ ความแปลกใหม่ที่สร้างขึ้นเพื่อให้ผู้เล่นสล็อตออนไลน์นั้นเล่นอยู่เสมอเพื่อให้ผู้ที่เล่นสล็อตออนไลน์นั้นได้เพลิดเพลินไปกับความแปลกใหม่ของสล็อตออนไลน์อยู่เสมอ
โดยที่กล่าวมาข้างต้นนั้น pg slot auto นั้นพยายามปรับปรุงอัพเดท และพยายามสร้างสรรค์เกมใหม่ ๆ ออกมามากมายเพื่อผู้เล่นโดยเฉพาะ และพยายามนำสิ่งที่อยู่ในกระแสมาสร้างเป็นตัวเกมสล็อตออนไลน์อีกด้วย ตัวอย่างเช่น pg slot auto ได้สร้างเกมทีมีชื่อว่า Crypto Gold ซึ่ง สล็อตเว็บตรง PG นั้นได้สร้างขึ้นเพราะช่วงเวลาดังกล่าวนั้น การเทรด Crypto กำลังเป็นกระแส และช่วงเกม NFT กำลังดังนั้น ถือได้เลยว่ามาแรงแบบสุด ๆ เหรียญ Bitcoins นั้นก็เป็นสิ่งที่คนคอยตามหาเป็นอย่างยิ่ง และพยายามขุด ค้นหา พวกเหรียญ Crypto เหล่นี้กันเป็นจำนวนมาก Bitcoins ณ เวลานั้น ตัวเหรียญได้ทำราคาขึ้นไปสูงถึง 2,200,000 หรือเทียบเท่าเงินดอลลาห์ 68,000 เหรียญเลยทีเดียว และมีผู้เข้าร่วมมากมายเพื่อหาซื้อเหรียญนี้มาครอบครอง โดยที่ pg slot auto เห็นความสำคัญและเห็นถึงความสนุกสนานที่สามารถเอามาปรับให้อยู่ในรูป
แบบของ สล็อตเว็บตรง PG ได้ และทำให้อยู่ในรูปแบบของการตามล่าหาสมบัติ Crypto และสล็อตเว็บตรง PG ก็มีการแจก Jackpot ที่สามารถทำให้ผู้เล่นสล็อตออนไลน์ได้เพลิดเพลินกัน
Unknown said…
post... thanks For Sharing !!Great information for new guy like Bhagwat Geeta in Hindi PDF
Unknown said…
post... thanks For Sharing !!Great information for new guy like anuvad kise kahate hain
slotpg said…
รวมทุกค่ายที่นี่ที่เดียวน้องๆสาวสวยน่ารัก ยินดีให้บริการเว็บมั่นคงและปลอดภัย แตกหนักแจกจริง มารวมสนุกไปกับเรา สล็อตเว็บใหญ่ PG สล็อต เว็บใหญ่ pg #แค่คิดถึงเราเงินก็อยู่ในบัญชี

Popular posts from this blog

SF Eating: Erics

Walk On