Sunday 21 July 2013

Travel Memoir : Elephant Safari at Kaziranga, Assam, Feb '13


The Portrait of a Morning Elephant Safari

IMG_0702

It’s a nippy February morning and the fragrance of wet grass hangs heavy in the air. A soft, balmy winter sun appears from behind the eastern hills and imparts a golden hue to miles and miles of elephant grass that spread out to as far as our eyes can see. A flat sheet of mist stretches across the horizon; the hills and the tree-tops seeming to levitate above it.

We sit perched on the elephant saddle swaying side-to-side, lumbering across the flat plains. We crane our necks, tug out our binoculars and keep our cameras handy – all with the eager expectation to spot the legendary Indian one-horned rhinoceros; we have all come here to see.

As we trudge through the wild grass, we are accosted by variety of deer – swamp deer, hog deer, sambar – some of them curiously gaping at us. A click or two, and we decide to move on.

I strain my ears to hear the sounds: the chirp of birds, the occasional low-decibel rumble of the elephant we are riding, the faint rustle of grass, – and also the murmur of the elated tourists riding on elephant-back at a distance. The surroundings seem blissfully tranquil as the lazy forest wakes up. I try to take in as much of it, hoping if I could take some of it back home.

IMG_0714We turn around a thicket and the mahout riding the other elephant makes a gesture to ours. Something seemed to have shot into sight, he gesticulated with his fingers! Our mahout tugs at Padmini, his elephant and daughter, and we make a quick dash. Suddenly, a full-grown adult rhinoceros comes into view. Majestic, solitary and somewhat grotesque – the Indian one-horned rhinoceros was once abundantly found across the entire Indus-Ganga-Brahmaputra basin, but excessive hunting reduced their natural habitat extensively. As shutter bugs go on a clicking frenzy, the animal becomes oblivious of our presence, turns towards us for a brief moment and then quietly walks away into the dense foliage. We get the hint: the animal doesn't want to be disturbed!

A mixed sense of excitement and contentment overcome the chattering tourists: we have finally spotted the legendary one-horned rhino in the wild. Our pachyderm Padmini marches on. By now, the sun is well above the horizon, and paints a pale yellowish-green glow to the carpet of elephant grass. Our mahout points out how the entire swath of wild grass will be burnt down in March to make way for new ones during monsoon – and how during monsoons, the entire flat plains get flooded by the raging Brahmaputra (it helps to know that Kaziranga is flanked by the Brahmaputra river to the north).

The Flora, Fauna and Forest Folklore

Kaziranga is a beautiful park and a nature’s bounty. Spread over 440 sq. km, it has 2/3rd of all surviving one-horned rhinoceros, the world’s largest number of white-water buffaloes and the world’s highest density of tigers. Yes, you read it right – the world’s highest density of tigers! We couldn’t spot any during our stay but hardly matters. Kaziranga is a jewel in the crown amongst all forest reserves in India in terms of its rich biodiversity and effective conservation efforts.

Its flat terrain, wide vistas, the hills (i.e. Karbi-Anglong) along the eastern and southern fringes, blossoming silk-cotton trees, numerous water bodies and crisscrossing rivulets make it more picturesque than other national parks

I have seen so far (i.e. Corbett, Rajaji, Silent Valley, Bandhavgarh, Betla). Anywhere within the park, it offers over 4 sq. km of expansive view. That makes it more breathtaking than many of the dense, moist deciduous forest reserves in India.

In my opinion, there is one more thing that makes Kaziranga different – they let you be! Tiger reserves in other parts of the country have become breeding ground for deliberated jungle folklore – of man-eating tiger tales, of disappearing cattle, of creepy encounters, so on and so forth! All this sensationalism is fed by forest officials and guides to eager tourists, ever willing to dole out more money or make unwarranted compromises in the hope of spotting the elusive tiger.  Kaziranga, on the contrary, feels more natural – stories narrated only if probed, less obstructive forest officials (we did a 3.5 hour jeep safari deep into the wild), and never an effort to extort money for any game-spotting. You will be somewhat taken aback by how well some of the seemingly-reticent safari guides know about the flora and fauna; so do make an effort to strike a conversation with them.

IMG_0687The Wild Grass Lodge

The car swerves to the right and takes a sudden detour from the NH37. The narrow path, lined by bamboo hedges and low-rise thatched houses, takes one by surprise. In a minute or so, a beautiful quaint lodge comes into view. With its warm wood-paneled flooring, large airy windows, stuffed deer, high sloped ceilings with ventilators and period furniture, Wild Grass evokes the unmistakable reminiscence of old-world British-era hunting lodges. So authentic in its ‘hunting-lodge’ feel that in spite of being told that it isn’t, my mind refuses to register.

The food on the menu card is quintessentially Chinese or mainstream Indian to cater to, seems like, “popular demand”. We insist on authentic Assamese food, though – Fried Banana Flowers (highly recommended), Smoked Fish ‘Paturi’ (i.e. in Banana Leaf; good), a local Pork Curry (average) along with rice and some yellow lentils. The food is wholesome and very mildly spiced. The staff is courteous but somewhat shy and slow. If you have to try some local dishes and flavors, you must insist on the waiter!
Wild Grass is an amazing place to stay – calm, relaxing, reasonable, good food and full of character. It is close to the Kaziranga central zone, which I believe has some of the highest animal sightings. There are other resorts closer to the Eastern and the Western zones. It might help to do a bit of research beforehand to find out which of the zones have seen the highest tiger and rhinoceros sightings in that season, and plan one’s stay accordingly.

Travel Memoir : Kedarkantha Summit Trek, Uttarkashi, Garhwal Himalayas. March '13




NOTE: All distances with respect to Dehradun. All timings are Indian Standard Time (IST)

It began with a disturbing, rickety overnight bus ride to Dehradun. The feeling that something that begins in a wrong way, would continue to be ominous all along, engulfed my tired mind as I stared outside the giant bus window. “How would Kedarkantha, our first major Himalayan trek, come along?” was the question I kept asking myself.
Turned out, it would be quite the opposite.
A hurried, anxious arrival at Dehradun
March 26. 07:30 am IST
The dash to the predetermined congregation-cum-pick up point outside the colonial-era Dehradun railway station was a mix of excitement as well as anticipation.
Meeting your fellow hikers for the first time who are absolute strangers is a peculiar feeling. Basis my past experience, I have come to realize that it’s them to whom one exposes one’s deepest vulnerabilities and insecurities, under trying conditions high up on the mountains.
Naturally so, I felt shy, nervous and try exchanging glances (am an amateur trekker after all)
We quickly exchange pleasantries, and are lobbed into Mahindra jeeps that immediately leave for Sankri, the base of our trek.
Dehradun – Sankri: The Yamuna I never knew existed
March 26. 07:30 am - 17:00 IST
Early morning drive through mountains can be quite a soothing experience.
No sooner do we leave Mussoorie (35km), than we are accompanied by a raging torrent far below at the base of the valley. We are told that it’s the Yamuna River, and that it would be our constant companion for the most part of the journey till Sankri village.
For a moment we could not believe our eyes. Crystal clear and gurgling hundreds of feet beneath our winding road, I was reminded of the appalling state of the same river Yamuna, we are witness to at Delhi.
The 192km distance from Dehradun to Sankri takes almost 8 hours to cover. The first leg of the journey is regular terrain found in the lower reaches of the Himalayas. The landscape, however changes remarkably after one crosses Purola (129km).
The air becomes cooler and the flora changes from deciduous to alpine. Purola is incidentally also the last mid-sized town on the road. Like all hill towns, Purola is also a barrage of settlements organized around one main thoroughfare. A  State Bank of India ATM, a smattering of provision stores, a confectionery shop, pharmacy, a roadside dhaba ('Hotel Mehar'), a liquor shop and rows of tiny settlements line the road.
Since we are told, this is the last civilized settlement, which in today’s mobile world implies cellphone connectivity. We make quick calls to our kith and kin, steal a quick lunch of rajma-chawal and proceed.
River Tons comes into sight as we prepare ourselves for the last leg of the winding journey to Sankri. Tall pine trees, dense woods, gurgling streams and occasional glimpses of distant snow-covered mountain ranges behind awning valleys accost us. We have arrived in the Himalayan kingdom, at last!
After almost a 9 hour ride, we arrive at Sankri at about 17:00.
Sankri: A sleepy hamlet in the lap of high mountains
March 26 Evening.
Sankri is a small and sleepy hamlet that sits in the lap of massive snow-clad mountain ranges, deep within the remote Uttarkashi district. It is the last village on this route connected by bus; has no phone connection as yet and had no electricity until very recently (i.e. 2007). Located 13km inside the Govind Pashu Wild Life Sanctuary, Sankri is the gateway to the pristine Har-ki-Dun valley and the ethereal Swargarohini Peak (6096m) that towers beyond it.
IndiaHikes has made fairly decent arrangements for us. In this remote land, a hotel, Swargarohini Palace and a Garhwal Vikas Mandal guest house, our rest-houses for the night, seem like luxor to our tired minds and bodies.
A quick briefing session for the trekking group, quiet dinner at a dimly-lit Garhwali household, and a short after-dinner stroll under a sky awash with moonlight, and we are ready to retire for the night.
Our rooms have 5 wooden cots each, cotton blankets and washroom with running water – bare necessities we take for granted, that'll become no less than luxuries for the next 3 days to come!
Sankri (6300 ft)– Juda Ka Talab (9200 ft): a climb through dense pinewood followed by snow sections
March 27. 07:30 am - 12: 15 pm IST.
A beautiful, sunny morning beckons us. The snowy mountains, painted golden-yellow by the rays of the sun, seem way too inviting. We freshen up, hang our backpacks, dig into a quick breakfast and prepare ourselves for the day ahead.
there's excitement in the air! At 07:30 am sharp, we commence walking, almost in tandem. A short 10min walk on flat road and we reach the base from where the climb starts. It's a decently moderate gradient.
Almost immediately, we are in the midst of a dense forest. Pine, fur, birch, spruce, and occasionally Maple – trees, some almost 100 feet tall – provide the much appreciated canopy.
We keep climbing, almost relentlessly. The group splits immediately, into – the racers: who determinedly pace ahead, as if it was a competition; the steady walkers: who walk at their own pace, occasionally stopping to catch breath; and the strollers: who gambol about, drifting, laughing, posing, clicking, and trek, when they find time for it. Some, of course, are perennially breathless. They are a different breed.
The walk to Juda-ka-Talab is very picturesque, indeed. You are in the constant shadow of the Swargarohini and Kalanag peaks (literally translates to ‘Black Peak’). Swargarohini, lying at the head of the cradle-shaped Har-Ki-Dun valley, has a mythological legend associated with it: The Pandavas and Draupadi had escaped to heaven from this peak, after the Great Mahabharata War.
The route is adorned with pretty bright red flowers called ‘Burans’. Part of the Rhododendron Arboreum family, this evergreen shrub, we're told, produces very nutritious juice and wine. We pluck a flower or two, and move on.
The entire trail till you hit snowline is carpeted with dry leaves, at least in March. Walking on dry leaves emanates crunchy sound, that combined with the gurgling of tiny rivulets, the very occasional chirp of birds and faint rustle of pine leaves, immensely relaxes one's strained nerves.
We hit snowline at around 8000ft, roughly around 10:30am. We pause at a clearing surrounded by Maple trees, to catch breath, steal some shots and begin our onward journey soon. As we climb higher, the view gets better and better. The receding tree-line ensures that we get wider panoramas: Har-Ki-Dun to North-East, Bandarpoonch range that towers over us and Rupin Pass, and the mountains of Himachal Pradesh beyond, to our North West.
It feels like no less than heaven here.
Huffing and puffing
Dragging ourselves up the icy slope
Prodding and digging
Trekking poles, our rays of hope
We stare above
No end, seemingly, in sight
But, when we halt to look around
The universe here seems just so right
At 12:15pm, roughly 4:45 hours after we had set off, we arrive at the Juda-Ka-Talab camp (9,2000 ft).
Juda Ka Talab: the campsite painted as if on an oil canvas
March 27, 12:30 pm - March 28, 07:00 am IST.
A half-frozen pond with moss-laden water, cocooned comfortably amidst tall conifers, and little camps set up on the lakeside escarpment, seems like the perfect campsite setting created straight out of an oil canvas. Juda-ka-Talab, here we arrive!
Our IndiaHikes team had already made the necessary preparations by the time we arrived. We are greeted with cups of fresh fruit juice on arrival. We offload burden from our tired shoulders, and settle at the campsite for the rest of the day.
Camping in the wilderness has different meanings for different people. Some stay glued to the campfire, warming themselves and their paraphernalia; others ambling about the campsite, chatting and joking; some others stay by themselves, pretty much in quiet reflection, and a few, retire to their tents.
As evening drew to a close, the ambient temperatures fell drastically to -1 degree Celsius. Our trekking group huddled closer to the camp fire, exchanging stories, belting out old Hindi and English numbers, or simple lazing around. Sometime before sunset, our coordinators, Manish and Rajmohan demonstrated techniques for walking comfortably on snow and rescuing co-trekkers should any calamity strike.
As darkness fell, we hurriedly finished our dinners (meals organized by IndiaHikes was quite wholesome, delicious and rich in nutrition), and scampered to our little tents.
Juda Ka Talab (9,200ft) - Kedarkantha Base Camp (11,200ft): the day we mastered snow
March 28, 07:30 am - March 28, 11:30 am IST.
 The 2000 ft climb to the base camp was entirely a snow climb. The trail that led out of Juda-Ka-Talab was initially a winding snow-filled path amidst tall trees. Further ahead, we encountered a reasonably steep section, completely snowed under, that required traversing on some sections as the trail was too steep the walk straight upwards.
We made good progress and reached a lower ridge in about 2 hours.
Once on the ridge, the walk became reasonably easier. With each step, the magnificent mountain unfolded in front of us, in all its hues and vistas. We felt humbled and awe-inspired. As intimidating as it seemed, climbing the mountain one step at a time felt like life's little lessons well learnt: no obstacle, however big, is insurmountable. All one needs to do is conquer it one step at a time.
Out on the ridge, we suddenly felt the brute force of the wind chill smashing into our frigid faces. We looked around; we could spot snowstorms lashing the higher reaches of the Bandarpoonch range, which now seemed remarkably closer to us.
Our trek coordinator Manish yelled out to us to pace up as the grey clouds were fast approaching us. A sudden rush of blood and panic engulfed us. The gail grew stronger, we lowered our backpacks and yanked out our raincoats. Braving the tearing force of the wind, we somehow managed to slip into our raincoats, slithering and swaying. And suddenly, there came from the sky above, a downpour of soft, fluffy snow flakes. Snowfall, at last.
A wide expanse of snowy slope lay ahead of us. We paced up amidst diminishing visibility, then found a comfortable position under a tree and waited for the rest of the group to catch up with us.
What a mesmerizing and exhilarating experience it was! Indeed, the first ever snowfall for quite a few of us. In a moment the gale subsided, the snow receded and we proceeded towards Kedarkantha base camp, barely 15min away.
It almost seemed God had ordained a snowfall for many of us to witness it for the first time.
Kedarkantha Base Camp (11,200ft): no man's land
March 28, 11:30 am - March 29, 07:30 am IST.

Travel Book Review : "From Heaven Lake - Travels through Tibet and Sinkiang" by Vikram Seth

In a world (we inhabit) where travel restrictions have eased immensely and where technology enables us to
plan travel on our fingertips, it’s hard to imagine what it could be like to travel through interior China and Tibet during post-Cultural Revolution days. And this is where Vikram Seth’s book takes the readers to!

A young and curious 23 year-old lad, a student of Nanjing University, Vikram, tears away quite by chance and choice from the suffocating mold of his 3 week-long “organized” tour group and ventures into an impossibly difficult part of the world.

His journey commences at the brazenly hot military outpost town of Turfan, in extreme North-Western desert province of Xinjiang. Over the next 3 weeks, he weaves a beautiful tapestry of people, places and experiences he comes across; memories that are sure to live with him for a lifetime.

He visits the most fascinating of places – the hot oasis town of Turfan, the historical Xian and the erstwhile capital of China, the ubiquitous Sun-Yat-Sen Parks he confronts in each town wherever he visits, Liuyuan – the treeless, dusty rail junction and transport yard that is incidentally also the gateway to Tibet, nondescript remote towns of Dunhuang and Nanhu, the wilderness outpost of Germu, the exotic wild-lands of Tibet, the imposing Potala and Norbulingka palaces at Lhasa and finally, an almost adventurous crossing over into Nepal across the mightily swollen Dudhkosi river.

Vikram makes astute observations everywhere he goes. Religion, cultures, historical references, politics, local mindsets, ethnography – you name it and he has it. One wonders how much he reads up about people and places before he actually visits.

To call it a travelogue, doesn’t quite do justice to the book. It is nothing short of a memoir – of one’s experiences while traveling through a remote land.


A quintessential book to read for any intrepid traveller.

Travel Book Review : "No Shortcuts to the Top" by Ed Viesturs

With a title like “No Shortcuts to the Top”, one easily mistakes it to be staid, self-help management
handbook. Till one catches a closer glimpse of the cover! Donned in red spacesuit-like apparel, an elated Ed Viesturs, the greatest American mountain climber, stands aloft with an air of accomplishment above the floating clouds below. No prizes for guessing the place, though.

The book opens to a spine-chilling account of a rescue operation, that almost turns sour, high on the slopes of savage K2 – the world’s second highest mountain peak. The reader, somewhat shaken by the distress of climbers almost swept away by an avalanche, comes face-to-face with the perils of high-altitude mountain climbing. Clearly, mountaineering isn’t for the faint-hearted.

As the reader turns the pages, tales of grit and determination (against all odds) emerge. In his quest to scale all the fourteen 8000 meter Himalayan peaks, Ed offers a peek into lives of legendary climbers he had happened to interact at close quarters. Reinhold Messner, Hermann Buhl, and Rob Hall –what drove them, how they inspired him, and in some cases, what led to their deaths.

One could think that much of the book would be about Ed – his Endeavour 8000, his climbing successes/failures, near-death experiences, and his personal chronicles. In fact, it is much more than that! What makes this book a great one is that it offers a glimpse into the world of mountaineering, albeit through his eyes.

Himalayan mountaineering may seem like ‘just another passion’ to most; but to many, it means much more. To nations sponsoring expeditions, summiting bestows national pride. To mountaineering gear manufacturers, the icy slopes of the Himalayas offer prime testing as well as marketing spots. To medical researchers, Himalayan mountaineers offer real-world evidences into the field of high-altitude medicine. To Gorkhas, it means putting their daily bread on the table. And, to climbers who risk their lives, an answer to the existential question of what gets them going.

Mountaineering, in all its pristine glory, brings with itself emotional struggles, personal sacrifices and expedition politicking, not to mention high risk. Every serious mountaineer battles the over-arching tension between passion and livelihood. During expeditions, climbers face the eternal conflict between personal summit aspiration and team goals. Expedition teams compete often leading to dereliction of duties and responsibilities. The biggest dilemma however is when a climber on a summit attempt confronts the dilemma of moving on or turning back, especially hours below the summit. And all too often, this decision often draws the line between life or death.

Ed continues to inspire the reader through every action and decision. He is no doubt a great mountaineer but also a fantabulous human being. His sagacity is reflected in his motto – “reaching the summit is optional, getting down is mandatory”. Simple as it may sound, one only realizes by reading his accounts how tough it is to abide by it. He also displays commendable sagacity in the face of extreme adversity, exemplified during the infamous 1996 Everest disaster rescue operations.

Beautifully woven and gripping, this book is replete with remarkable experiences, interesting anecdotes, copious insights and helpful tips. Stories of hope amidst despair and triumph against tragedy throw open more questions about the prudence of engaging in high-risk adventure sports such as Himalayan mountaineering.


In essence, a must read for every trekker and an aspiring mountaineer. 

Tuesday 16 July 2013

Technology : The Promise and Perils of Big Data


It is common knowledge that we are witnessing an explosion of data, particularly digital, in every facet of our economy and society. Synonymous with this phenomenon, the term ‘Big Data’ has increasingly gained relevance. Today, Big Data has reached a point of ubiquity where almost every business and technology journal, forum or research is talking about it.

What is Big Data? 

Simply put, “Big Data” refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze.

There are concepts that have become fundamental to defining big data. Commonly known as the “3V’s of Big Data1”, these have come to become the universally accepted parameters of big data:
1.     Volume – Volumes at which data poses to become a problem, and when it overwhelms existing standard solutions within the enterprise.
2.     Velocity – High velocity of streaming data, event data and transient data.
3.     Variety – Structured and Semi-structured data. Dynamic schema and non-relational data.
There is no universally accepted dataset size, velocity or variety benchmark that identifies one as big data. Different industries have different benchmarks, and with time, the needle is always shifting.
A University of California at Berkeley – IDC Research (sponsored by EMC) estimated that in 2010 enterprises globally stored 7 exabytes2 of data while consumers stored 6 exabytes of data in their PCs and Notebooks. By some reckoning, IBM claims that 90% of world’s data has been accumulated over the last 2 years alone. Consider this – by 2009, every company with 1000+ employees in the US stored 200 TB of data (twice the size of Wal-Mart's data warehouse in 1999).



What is the source of all this massive data? 

This volume and detail of information is being fuelled by data captured from scientific measurements and experiments (astronomy, physics, genetics, etc.), peer-to-peer communication (text messaging, chat lines, digital phone calls), broadcasting (news, blogs), social networking (Facebook, Twitter), authorship (digital books, magazines, web pages, images, videos), administrative activities (enterprise or government documents, legal and financial records), business (e-commerce, stock markets, business intelligence, marketing, advertising), and more. Data coming in from multiple sources could possibly be incomplete, ambiguous and even, inaccurate. And this is just the tip of the iceberg! As per Cisco estimates3, there were 8.7 billion connected devices in 2012, there’ll be 15 billion by 2015 and 40 billion in 2020. And as billions of physical devices get connected with networked sensors, the ‘exhaust data’ (i.e., data that are created as a by-product of other activities) from mobile phones, smart energy meters, automobiles, industrial machines, aero planes etc. will throw forth terabytes of data in the age of Internet of Things4.

What are the Promises of Big Data?

1.     Savings: Traditional analytics has been a costly affair. Licensing fees and costly upgrades have stymied the use and spread of analytics. Big data changes all that. The sheer brute force of powerful algorithms has resulted in time savings – in organizing, analyzing and presenting data. This is over and above reduced software licensing cost/operation cost either from ETL tools, data archiving etc. Intel has already made significant strides in this direction.

2.     Data-as-a-Service: Big data has led to the creation of a new business model/service within enterprises where data is collected from multiple sources in order to make them available for consumption. The primary focus at this function is to defining entities and collecting raw data. One may think of it something like a data library. This concept was not possible before. Higher-volumes of transactional digital data is helping enterprises with providing a more accurate and detailed performance information from exposing & analyzing variability to improving performance through better management decision-making.

3.     A platform for data mix and match: Big data helps find insight across multiple transactional data. New types of data like logs, sensors, social media feeds etc. are now being analyzed over and above the traditional tabular data. It is this form of analysis that wasn’t possible before.

4.     Customer Segmentation: One great example of big data is differentiating between customers in a more meaningful way. More precisely tailored products or services. Example, Tesco5, the European retail giant, taps its loyalty program to collect customer purchase intelligence, which it then analyzes to inform a variety of decisions, including the micro-segmentation of its customers and improving promotions that ensure 30% fewer gaps on shelves. Segmentation could also help government agencies and politicians through higher-quality, customized civic engagement (as was implemented during Barack Obama’s Presidential campaign)

5.     Replace human decision-making with automated algorithms: Sophisticated analytics can substantially improve decision making, minimize risks, and unearth valuable insights that would otherwise remain hidden. Example: Tesco5 uses models to understand effect of discounts on sales. By intelligently discounting prices (i.e. not too early), Tesco raked in 30million pounds ‘pure profit’.

6.     New Products, Services or Business Models: Big data can be used to improve the development of the next generation of products and services. For instance, manufacturers could use data obtained from sensors (embedded in products) to create innovative after-sales service offerings such as proactive maintenance, those in healthcare could accelerate the development of new drugs by using advanced analytics, and automobile firms can create new, proactive after-sales maintenance service for automobiles through the use of networked sensors.

7.     Delivering high performance applications: Big data provides alternate capabilities using NoSQL database which primarily focuses on application specific transactions rather than be driven by the underlying data model.

What are the Perils in Big Data?


1.     Overlooking data veracity: Over years, enterprises, institutions and governments have built massive datasets. However, much of this development has taken place in an environment replete with ‘silo-ed’ systems, poor processes and inconsistent methods of data input. The result – mounds of incorrect, imprecise, duplicate and in many cases, uncertain data. According to a recent Experian QAS® study6, 36% of U.S. marketing organizations interact with customers and prospects through 5 or more channels, 94% of businesses suspect their customer and prospect data might have inaccuracies, and on average, as much as 17% of information in cross-channel marketing databases is believed to be wrong. Gartner7 predicts that poor data quality effects overall labor productivity by as much as a 20%. No big data initiative can be fruitful without data veracity. This is the single-most biggest risk and impediment to big data initiatives.

2.     Using untreated data: Traditional DW/BI architecture has spent considerable resources on ETL/MDM but considering the variety and velocity of big data, automated data preparation and cleansing tools are still very premature8. Yet, any attempt to use big data platforms as a container to load data without any types of classification, categorization, entity definition etc. will only result in failure. A metadata is also an absolute prerequisite to derive desired value from big data.

3.     Implementing big data in ‘silo-es’: The silos will not only look at ‘local’ problems, it may also not use the potential of data mix ‘n’ match and thereby, loose the benefits of big data in terms of analytics, data as a service etc.

4.     Not knowing your problem well beforehand: Expecting every problem can be solved using big data is stupidity. Not knowing the problem and yet expecting big data to solve it, is setting up for failure. Agreed, data science is supposed to answer unanswered questions, but still there are some boundaries. To know the unknown without any boundaries will take significant time as boundaries need to be defined first. This is why initial implementation of big data is proving to be time consuming and costly, and enterprises should be ready for that.

The data to information to actionable insight value chain is tougher than one can imagine. The problems of heterogeneity, scale, timeliness and complexity remain. For instance, most online data is unstructured, and little vale can be derived unless data items are appropriately ‘linked’. Data analysis, retrieval and modeling are the other foundational challenges. Given the scale of data and underlying algorithms, analysis has hit a bottleneck. Finally, the presentation of the results and its interpretation by non-technical experts has proven to be a major impediment to successful big data implementations.

Besides this, when you search for patterns in very, very large data sets with billions or trillions of data points and thousands of metrics, you are bound to identify coincidences that have little or no predictive power - even worse, the strongest patterns might be:
o    Entirely caused by chance (just like someone who wins at the lottery wins purely by chance),
o    Not replicable,
o    Having little predictive power, but obscuring weaker patterns that are ignored yet have a strong predictive power.

As a matter of fact, the difficulty in realizing value has perhaps become one of the biggest challenges to big data implementations.

5.     Insufficient focus on addressing the talent gap: Much has been said and published about the looming talent gap in the area of big data. McKinsey has projected a 50-60% shortfall in the US by 2018. Gartner has also echoed similar sentiments stating only one-thirds of 4.4 million big data jobs will be filled in by 2015. Unlike analytics, big data requires Data Scientists who bring together a very diverse set of skills: deep business insights, data visualization, statistics, machine learning and computer programming. Policy makers have to play a significant role in mitigating this talent shortage through education and immigration policy.

6.     Analytical overkill, cognitive bias & human limits: Business decisions – to invest/not to invest, to retain/to let go off, so on and so forth – are based on human judgment. And judgments are contextual, social and value-led. This is where computer-driven data analysis-led approach might fail. Not only does data fail to capture context, enormous data could lead to false hypotheses and cognitive biases – the falsity of which just grows with more data9. It pays to keep in mind that data analysis can yield ‘false positive’ signals – because of the choice of statistical algorithms, and the fact that the interpretation (of the results) is done by those who lack expertise.

7.     Data security and privacy (and the risk of misinterpretation): Digital ‘breadcrumbs’ we leave behind as we go about our everyday lives create a trail of behavior that is not only followed, captured, stored and mined “en-masse”, they are also discreetly thrown into machines, and made to run on seemingly workable algorithms – all with the purpose of identifying correlated phenomenon. The issue of privacy (and the associated risk of misinterpretation) is scary and throws open many uncomfortable questions – Who decides if an algorithm is fool-proof? Who determines if a seemingly ‘predicted’ behavior mirrors an actual one? What defines “fair” use of data? Who possesses the legal rights to ‘mine’ your data? Who speaks for us? Who is responsible when an inaccurate piece of data leads to unintended – or worse still, negative consequences?

Like any emerging technology area, big data too faces its share of systemic challenges. Despite this, sectors like computer, electronics, information technology, finance, insurance and government sectors are already making headway into deriving competitive advantage out of big data. Yet, there is a long way ahead. Policies related to privacy, security, intellectual property, and even liability will need to be addressed by governments and policy-makers. Organizations will need to not only to put the right talent and technology in place but also structure workflows and incentives to optimize the use of big data.

References:

Research Papers:

“Big Data: The next frontier for innovation, competition, and productivity.”
McKinsey Global Institute Research Report. Dated: June 2011

1 Doug Laney, Gartner Analyst. “3D data management: Controlling data volume, variety and velocity.” MetaGroup Research Publication. Feb 2001

2 One exabyte of data is the equivalent of more than 4,000 times the information stored in the US Library of Congress.

3 Dave Evans. “The Internet of Things: How the Next Evolution of the Internet Is Changing Everything.” A Cisco Internet Business Solutions Group (IBSG) Research Paper. April 2011.

George A. Miller, “The magical number seven, plus or minus two: Some limits on our capacity for processing information,” Psychological Review, Volume 63(2), March 1956: 81–97.2

Ted Friedman, Michael Smith. “Measuring the Business Value of Data Quality”. Gartner Research Report. October 2011

Cisco's 2012 Visual Networking Index (VNI) Forecast.  

McKinsey & Company’s Business Technology Office report in 2011

Internet Articles:

“Big Data and Government Transparency” Applied Data Labs (A Data Technology Research and Advisory Labs). Click to read

4 “How Many Things Are Currently Connected To The "Internet of Things" (IoT)?”. Contributor: Rob Soderbery, Cisco Executive. Click to read

5Tesco uses data for more than just loyalty cards”. Paul Miller, Contributor, Cloud of Data. Dated: October, 2012. Click to read

6 “Poor quality data can hurt cross-channel marketing efforts” Erin Haselkorn, Experien QAS® Marketing Services. Click to read

7Data Veracity”. Michael Walker, Data Science Central. Dated: Nov 28, 2012. Click to read

8 “Realizing Big Data Benefits: The Intersection of Science and Customer Segmentation”. Contributor: Neil Blehn, Wired Insights. Dated: June 7, 2013. Click to read

9 “What Data Can’t Do?” Dadiv Brooks, The New York Times. Dated: February 2013. Click to read
“The Hidden Biases in Big Data” Kate Crawford, Harvard B