Maxing out Multicore
From smartphones to data centers, multicore processors are becoming the norm. The extra processing power is good news for developers, but there's a catch: The real-world performance gains are often held back by old ways of thinking about coding, a caveat summed up in the oft-overlooked Amdahl's Law.
More than two years ago, University of Wisconsin Professor Mark Hill gave a presentation about how Amdahl's Law affects multicore performance. He recently spoke with Intelligence in Software about why parallelism is key to unlocking multicore's benefits and why the computing world could use a new Moore's Law.
Q: In your presentation, you mentioned a conversation you had in a bar with IBM researcher Thomas Puzak. He said that everybody knows Amdahl's
Law but quickly forgets it. Why?
Mark Hill: You learn the math of this, but often when it comes to real life, you forget how harsh Amdahl's Law really is. If you're 99 percent parallel and 1 percent serial, and you have 256 cores, how much faster do you think you can go? You'd think you get a speed-up of maybe 250 out of 256. (A speed-up for 250 means that one is computing at 250 times the rate of one core.)
But the answer is a speed-up of 72. That 1 percent has already cost you that much. That's the kind of thing I mean. People's intuition often is more optimistic than if they did a calculation with Amdahl's Law.
Q: Hence your point about why there's a growing need for dramatic increases in parallelism.
M.H.: Correct. It also ties in with the fact that I don't think you're going to take old software and get dramatic parallelism because it's going to get that sequential component down.
Let's say you get the sequential component down to 35 percent. Then your speed-up is limited to three -- at most three times faster than a single core. Nice, but hardly what you want. So in my opinion, dramatic gains in parallelism are going to have to happen due to new software that's written for this person.
Q: For developers, what are the challenges to writing parallel-centric software, for lack of a better term? Is it mainly a change in mindset?
M.H.: It's a pretty huge hurdle. There have been people writing parallel software in niche domains such as supercomputers, but most developers don't have experience with it. If you think of software as a numerical recipe -- you do this, you do that -- parallel computing is like a bunch of numerical recipes operating at the same time. That can be conceptually a lot more difficult. It can be easier if you have a large dataset and say, “Let's do approximately the same thing on each element of this large dataset.” That's not so mind-blowing.
Part of the problem is that the literature can be a little biased because you can more easily publish results that show things working fantastically than working poorly. If you read these papers, you might think things are working pretty well, but people select things they want to publish with as opposed to the problems need to be done.
Q: You've talked about the need for a new Moore's Law, where parallelism doubles every two years.
M.H.: It used to be that you could design a piece of software, and if it ran like a pig, you could say, “Well, it's not a problem because processors are going to get twice as fast in two years.” Going forward, that's going to be true only if you get parallelism that keeps increasing. That's not going to happen by luck. You're going to have to plan for it.
Q: Maybe there's an analogy with oil: When gas prices skyrocket, as they did in the 1970s and again today, automakers start looking for ways to wring every mile they can out of a gallon. In the computing world, enterprises want data centers that aren't electricity hogs, and smartphones that can last an entire workday before they need a charge.
M.H.: Electricity is increasingly becoming the limiting factor in machines. In this new era, there's going to be a lot more pressure for the code to be more efficient. People say, “That doesn't matter anymore because computers are so fast.” Yeah, but if my code is twice as efficient as your code, that's going to matter because if somebody's battery lasts twice as long, it's a very good thing.
To download the slides from Hill's presentation, visit CS.Wisc.edu
Photo Credit: @iStockphoto.com/Petrovich9
- Top 11 Technologies on the Verge of Extinction
- The Top 5 Movie Phones
- Why You Need Double Protection Now: Hardware and Software
- The Long-term Commitment of Embedded Wireless
- Maxing out Multicore
- Google's New Airfare Search System
- The Latest Buying Gimmick: 'Google Wallet'
- Google Plus Now Open to All
- Facebook and Google Build DIY Servers
- Half of Global PC Users Use Illegal Software
- 4 Ways to Save on Gadgets
- Cell Phone Radiation: 5 Ways to Minimize It Now
- How to Choose the Right Tablet
- 5 Cool Apps for Your Labor Day Weekend
- 4G LTE: Not So Fast
- What Can the Cloud Do For You?
- Ubiquitous Computing: Is Ubicomp at a Tipping Point?
- Strike Back at SQL Injections
- The Mob That New Technology Has Made
- Social Networking Leads to Smoking, Drinking and Drug Use
- Cloud 101: Are You a PC, Mac or Cloud?
- Can You Get Fired for Your Blog Posts?
- Should You Switch to Google+ ?
- Navigate the Booming Computer Science Market
- China Outpaces United States in PC Market
- How Apple's iCloud Changes Business
- Use the Cloud to Go Mobile
- Apple iPad 2: A Smarter Business Tool
- Why You Can't Ignore iPhone / iPad Security
- Should You Let Your Teens Blog?
- Should You Talk Politics Online?
- How to Watch TV for Free -- or Close to It
- The Cost of Paying for Netflix
- Future for Slates, Tablets and iPads
- Social Media and Privacy (or Lack Thereof)
- Small Business Video: Life After the Flip
- Is It Safe to Post Photos Online?
- 9 Ways to Avoid Online Scams
- LinkedIn Offers New Options for Students
- 4 Strategies to Avert Virtual Arguments
- Cloud Computing 101: Protect Yourself Online
- How to Recover From a Social Media Mistake
- Why You Should Inventory Business Cloud Use
- Emerging Technology Has Positive Impact in Classroom
- 4 Apps to Manage Your Contacts
- Portable Tech Gadgets You Need This Summer
- Best Tablets for Your Business
- Security Tips for Your Smartphone or Tablet
- 5 New Cloud Tools You'll Love
- Can a Tablet Replace Your Laptop?
- Why Wireless Needs a Network of Networks
- 5 Steps to a Successful Enterprise Wireless Strategy
- 5 Keys for Moving Enterprise Security to the Cloud
- Tips for Building and Deploying Cloud-based Apps
- 5 Business Lessons You Can Learn From Mark Zuckerberg
- Russia: The No. 1 Base of Global Internet Attacks
- Researchers Say New Botnet TDL-4 Poses Big Threat
- The Internet Grows Up
- Ten Ways to Keep Your Online Information Secure
- How Facebook and Your Free Time Can Get You Fired
- The Only 10 Android Health Apps You Need
- Connected TV
- Near-Field Communication Technology the Next Big Thing
- The Future of 3-D Video
- Is Your Teen Sexting?
- Should You Ban Your Tweens From Facebook?
- Home Movies: Then and Now
- Is Social Media Malware Infecting Your Business?
- Is Your Tween (Illegally) on Facebook?
- Are Free Public Wi-Fi Networks Safe?
- Explosion of Creativity: Power of Online Communities
- The Future According to Google
- 5 Cheap Alternatives to Hiring a Personal Trainer
- When Your Dream Company is Hiring on Twitter
- Colleges Bring Campuses to Facebook
- Technology Powers Revolutions and Saves Lives
- Best Photo Apps
- Virtual World No Substitute for Real One
- Best Phone Apps for Busy Women
- How to Prevent Identity Theft
- How to Use Facebook So It Does not Use You
- Worst-case Computer Scenarios
- Google Chrome OS Notebook: A Security Game Changer?
- What Is Cloud Computing?
- How to Prevent Data Breaches
- Best Tips for Sharing Videos
- Do You Need an iPad for Your Small Business?
- The App Guide: 5 Must-have Shopping Apps
- Project Management Tips From the Pros
- Finding the Right Skill Set
- Who's Gawking at Your Photos?
- Dealing With Virtual Stalking
- CES 2011 Report - Consumer Electronics Show
- Time to Gear up for 3-D TV?
- How to Get the Best Service From IT Vendors
- Dating Apps: The Lowdown
- New Website Streamlines College-Aid Application
- Gift-card Resale Market Thrives Online
- Stop Cyberbullying Now!
- 5 Cloud Tools to Boost Your Productivity
- Mobile Pay Can Give You an Edge
- How to Find the Best Deals Online
- Should You 'Friend' Your Teens Online?
- Should You Really Post That Comment?
- 5 Smart Tactics for LinkedIn Self-Promotion
- How Repressive Regimes Use the Internet to Keep Power
- WikiLeaks: Diplomacy as Usual
- The Rising Threat of USB Drives
- Integrate Cloud Solutions With Caution
- Tech Solutions to Track Your Resolutions
- Crash Course in Computer Maintenance
- How to Pick the Right Cloud Provider
- Web Tools for Starting a Small Business
- 4 Life-changing Resolutions You Can Stick to
- Create Your Own Social Network
- What the Web Says About You
- FTC Chairman: 'Do Not Track' Rules Would Help Web Thrive
- 'Do Not Track' Rules Would Put a Stop to the Internet As We Know It
- Are Federal 'Do Not Track' Rules Needed?
- The Political Power of Social Media
- Top 3 Kid-safe Social Networks
- 5 Ways to Cultivate Your Kids' Online Reputation
- Preparing Our Children for Global Digital Citizenship Success
- The Great E-reader Roundup
- The NFL's Highest-scoring Apps
- New Gadgets Straight From the Big Screen
- A Game Plan for Protecting Stored Data
- The Drive for Real-time Collaboration
- Hang Onto People Who'll Help You Advance
- Leading New Developments in Visual Computing
- Meet Mr. Industrial: Justin Lassen's Music Machine
- Must-see TV on the Internet
- Share Safely on Social Networks
- 6 Essential Rules for Safe Online Shopping
- Five Steps to Kid-friendly Surfing
- High-tech Help for Weight Loss
- Create a Web Site or Blog for Your Family
- Video Eyewear for Sunglass Cinema
- Smart Phone Apps Help Stop Distracted Driving
- Online Computer Backup Services Remove Hassle
Copyright © 2011 Studio One Networks. All rights reserved.