Aligned Planets

Colin Charles: MOL at the center of online & offline payments

Planet LCA 2009 - October 22, 2012 - 17:15

There’s a chance that Malaysian payments will get shook up. From an online perspective, you’ve got the former nbepay becoming MOLPay. From an offline perspective, you’ve got a joint venture between softspace and MOL to form MOLCube (e27 cover it too). The center of all of this is MOL.

For me, I’ve been waiting for a softspace device for quite a few months. I was excited since April 2012. I was told a device would be coming my way from 18 June 2012, and never heard back; the presumption is that people are using this device according to their website. But it is not available for the “general population”.

I have never met Ganesh Kumar Bangah, the man behind MOL, but being a young CEO, I figure he’s got the chops & energy to pave the way. Besides, he’s backed by Berjaya tycoon Vincent Tan.

Ugliness begone, let there be better online & offline payments and this will pave the way for e-commerce as well as physical versions of e-commerce (pop-up stores, bazaars, heck, imagine your pasar malam vendor going online).

A lot of this will involve lobbying Bank Negara Malaysia (BNM). I don’t believe any payment gateway intentionally wants to provide terrible user experience, I believe its usually to feed regulatory requirements.

Looking forward to payments in 2013. It can only be better than today.

Related posts:

  1. The state of e-commerce payments in Malaysia: still terrible
  2. Offline GMail via Google Gears
  3. Square-like payment devices in Asia

Categories: Aligned Planets

Colin Charles: Didn’t take long for iOS6-only apps

Planet LCA 2009 - October 21, 2012 - 02:00

It didn’t take long for my prediction on 24 September to come true on 11 October. My suspiscion was also correct, it would be led by Marco Arment, but it had nothing to do with Instapaper, it was the launch of a new product titled The Magazine.

The reason to make it iOS 6 only?

It uses some iOS 6-only features and fonts, and it’s architected for iOS 6’s gesture handling. Setting this high baseline also greatly simplifies testing, maintenance, and future updates.

Wow. Fonts & gestures. Amazing.

Greed or tradeoff from Apple? 

Related posts:

  1. Apps are the new channels
  2. my pre-upgrade iOS6 thoughts
  3. Messenger apps revisited

Categories: Aligned Planets

Colin Charles: The state of e-commerce payments in Malaysia: still terrible

Planet LCA 2009 - October 20, 2012 - 20:14

Today I tried to checkout RM450 using iPay88. They only accept Visa or MasterCard credit cards, so I pulled my wallet out.

I thought I would use the Citibank card today. I got sent to an error page. So I clicked the back button to head back and thankfully this worked.

I dug further and found a Direct Access card. I have to choose an issuing bank and now had to think a little harder to figure out that this card belongs to CIMB. I was sent a code via USSD verification which was valid for a mere 3 minutes. I had to run to my phone which was charging upstairs and run back down to make the transaction.

Later I see an SMS from Citibank giving me my OneTime PIN that is valid for 4 minutes. I never even got the chance to use it.

Now, lets say I was the average consumer.

  1. What would I have done with the error page?
  2. How would I have reacted to seeing one design then seeing the iPay88 page? Seems close to an attack. Stripe doesn’t have this problem.
  3. How quickly would I have retried the same credit card before I gave up on the online purchase?

Anecdotal evidence from several online stores that I’ve been involved in suggests Malaysians are a patient bunch. They try up to three times for a credit card transaction before abandoning the cart. Some will email because its clear they really want the item. Most truly just give up.

E-commerce is slated to be big. But fixing payments should be crucial.

Related posts:

  1. 7-Eleven helps e-commerce in Taiwan
  2. Malaysia slugs credit card users who don’t pay up
  3. Is Lenovo Malaysia interested in selling their stuff?

Categories: Aligned Planets

Colin Charles: How to insult this blog

Planet LCA 2009 - October 20, 2012 - 19:46

Asking to buy advertising with a pitch such as:

Therefore I’m interested in buying advertisement space (my budget is unlimited).

Or worse, the following:

I would like to get an article on your blog about [REDACTED]. For this we would like to offer you 200 RM.

We are a [REDACTED] company and growing fast in all South East Asian markets. Once we get started we see the opportunity of building a long term relationship that would benefit both you and me.

We are currently handling a very handpicked selection of top blogs in Malaysia so the sooner you can get back to me the more I would be likely able to spend. Would be great if you can also send me your telephone number or skype account so we can have a quick chat on the details.

Do expect to get rather curt responses which tell you not to insult me.

Related posts:

  1. Quick notes: Monty Program Group Blog; Rename Maria
  2. Changes in the blog
  3. EducationaLinux and an interesting MSDN blog

Categories: Aligned Planets

Colin Charles: YEOLSIMHI haeyo

Planet LCA 2009 - October 19, 2012 - 18:32

NYT: The Thirst for Learning

YEOLSIMHI haeyo, Koreans say. Work hard. The phrase is spoken endlessly and serves both as a rallying call and a reminder that no one likes whiners. And no matter how hard a student is working, he or she can always work harder — or so goes the theory.

While I’m not a huge fan of the idea that a prep school starts at 7.40am and goes on right until 10.20pm, it is amazing to see the progress that South Korea has had in the past few decades. They’ve grown their GDP per capita almost three times over Malaysia’s, and it was only in the 80?s that they were still looking up towards Malaysia as a success story.

Remember to always take yourself and whomever is teaching you/speaking to you seriously. 

Just this week, I received feedback at a meeting and changed some slides for the next day’s presentation. The audience was impressed that I didn’t just say I’d take the feedback, but I acted on it by doing the necessary research in a limited timeframe. 

Remember to care. And be great.

Related posts:

  1. OLPC, by Jim Gettys
  2. Localisation and its merits
  3. Open Source Economy Conference 2008

Categories: Aligned Planets

Colin Charles: On being vulnerable

Planet LCA 2009 - October 19, 2012 - 18:17

FT: Time to open up at the office.

Vulnerability means opening yourself to hurt. And as hurt is something that hurts, opening yourself to it is something best avoided.

To risk getting hurt is brave. To act invulnerable is not.

The single most important difference between people who can connect and those who can’t is their willingness to be vulnerable.

Leaves me a lot to think about. I generally believe in having tall walls. Time to follow the work of Brené Brown clearly. 

Videos: The Power of Vulnerability, Listening to Shame.

No related posts.

Categories: Aligned Planets

Colin Charles: Jurisdiction, Internet law & alvivi

Planet LCA 2009 - October 19, 2012 - 11:46

Rais Yatim is at it again:

Speaking to reporters yesterday on the sidelines of an event, Dr Rais said, “We have legal redress under Section 233 and 263 of the Communications and Multimedia Act. But we would rather not use that first until and unless we get the results of what the Singaporean authorities are pursuing first.”

legal redress. i’m beginning to wonder, where does Malaysian jurisdiction fall when it comes to the Internet?

the alvivi blog was hosted on tumblr. last I checked, tumblr was not a malaysian company & has no presence in malaysia (this is quite unlike blogspot & google). tumblr is unlikely in singapore too. do malaysian or singaporean internet laws apply? can the communications & multimedia act 1998 be used just because these two are malaysian?!?

so what is alvin tan & vivian lee guilty of? recording videos of their sexual escapades. what’s wrong with that? content is king, and if they’re producing useful content, so be it. no one forces you to watch, or read, or be nosey, so you really can switch channels.

singapore isn’t jumping on this yet. they’re letting the university (nus) deal with the matter first. how is this the problem of the university? maybe its because at least one of them has a scholarship. universities shouldn’t dictate what students choose to do with their lives.

i’m not alone in thinking this is their business. after all with the billions of pages on the internet, there really isn’t much reason for people to cry afoul.

alvin is an entrepreneur at heart. i’m glad they’ve posted a response (seems vivian doesn’t do much talking).

they are after all collecting email addresses to build a mailing list. i’m old enough to remember jennicam. lifecasting is not a new idea either (iJustine,, etc.). imagine jennicam meets kink? there’s great production value here (of course, i have to admit i’ve not seen the content as the blog has been taken offline; but to get such traction, clearly it must be good to a selection of people). alvivi can clearly become a brand. 

obscenity laws? random acts? lets not curtail on the right to freedom of expression. remember, if you don’t like it, don’t view it.

Related posts:

  1. Keeping the (content on the) Internet relevant
  2. Piracy due to lack of legal options
  3. Blogger registration, revisited

Categories: Aligned Planets

Colin Charles: Why the mini iPad?

Planet LCA 2009 - October 18, 2012 - 13:16

Edwin Yapp thinks about why Apple would introduce a mini iPad. The thinking for me is simple:

  1. I almost exclusively use my Nexus 7 tablet now for everything. This includes surfing the Internet, reading books on the Kindle app, and more.
  2. The only thing it isn’t so good for at the moment is watching movies which I do on the plane (which I tend to be on a lot). And consuming video podcasts is definitely an issue since there is no iTunes syncing.
  3. The storage size isn’t so hot either – at 16GB I cannot load it up with a lot of movies like I can on my 64GB iPad.
  4. It is cheap. Replacing a USD$250+ device is much easier after 2 years than replacing a USD$700+ device when the software on it becomes obsolete.

Resolution size probably plays a huge role. I lug a 15? MacBook Pro around now, because I’ve always been using 15? laptops since the days of the PowerBook. Its simply because of the resolution: 1440×900. Today I’m thinking about a 13? MacBook Air because its lighter and it also supports the 1440×900 resolution. Going from 15? -> 13? is a smaller screen size with the same screen real estate. 

I expect that with all these HD/retina displays, you can just fit more onto a smaller screen size.

Many have assumed that iOS developers only focus on developing for 2 sizes which is untrue. 480×320, 960×640, and now 1136×640 just for the iPhones/iPod Touches. Then there’s the iPad’s at 1024×768 & 2048×1536. Why not get a third? :-)

A 7? iPad that syncs with iTunes, has more space than 16GB (maybe 32GB is the middle ground that I should probably grab), with a good resolution – its something I could definitely consider. Have to figure out how to watch movies on a smaller screen though…

Related posts:

  1. The iPad: Early-experience notes
  2. The iPad as a camera
  3. Dell Mini Inspiron? New Asus EeePC’s? Its the keyboard, silly

Categories: Aligned Planets

Mary Gardiner: Ada Lovelace Day: Marita Cheng, Robogals founder

Planet LCA 2009 - October 16, 2012 - 16:00

Today, October 16, is Ada Lovelace Day: write or record a story about a woman in science, technology, mathematics or engineering (STEM) whose achievements you admire.

This is a slightly updated version of a profile that has appeared on Geek Feminism and Hoyden About Town.

Marita Cheng was named as the Young Australian of the Year winner at the beginning of the year. She’s been involved in volunteering since she was a high school student, and in 2008, early in her undergraduate studies (mechatronic engineering and computer science at the University of Melbourne) she founded Robogals, which is an engineering and computing outreach group, in which women university students run robotics workshops for high school age girls.

Marita, while still in the final year of her undergraduate degree, is also an entrepreneur and has been previously awarded for her work as founder of Robogals, including winning the Anita Borg Change Agent award in 2011. In 2012 she travelled to several countries with the aid of the Nancy Fairfax Churchill Fellowship to study “strategies used to most effectively engage female schoolgirls in science, engineering and technology.”

While I have heard of Robogals, I hadn’t heard of Marita specifically before she became Young Australian of the Year. One of the fascinating things about starting the Ada Initiative is slowly discovering all the other amazing women who work in technology career outreach and related endeavours. But it’s a little embarrassing, judging from her bio, to have not heard Marita Cheng’s name before the beginning of the year!

Further reading:

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Australia.

Categories: Aligned Planets

Mary Gardiner: Ada Lovelace Day: Else Shepherd, leading Australian electrical engineer

Planet LCA 2009 - October 16, 2012 - 14:38

Today, October 16, is Ada Lovelace Day: write or record a story about a woman in science, technology, mathematics or engineering (STEM) whose achievements you admire.

Else Shepherd is an Australian electrical engineer specialising in communications equipment. She has co-founded multiple Australian engineering companies, including Mosaic Information Technology, a custom modems company, and Microwave & Materials Designs, developing microwave filters for mobile phones. She was appointed as the chairman of Powerlink, the state government-owned corporation maintaining Queensland’s high voltage electricity grid, in 1994, and has been a board member of the National Electricity Market Management Company (now known as the Australian Energy Market Operator).

Shepherd won Engineers Australia’s Peter Nicol Russell Memorial Medal in 2007, their most prestigious award, recognising an engineer with over 20 years of substantial contributions to professional engineering in Australia. As best I can tell, she is the only woman Peter Nicol Russell medallist. She is also a Member of the Order of Australia since 2003, and was the University of Queensland Alumnus of the Year in 2009. She is also a pianist and choral director.

Shepherd has talked about her experience as a woman in electrical engineering with University of Queensland publications. She and one other woman graduated in 1965, the university’s first women graduates in electrical engineering. She was unable to attend Institution of Engineers meetings in the 1960s, because they were held at the local Men’s Club. She continues to promote workplace flexibility, having used part-time work during parts of her career to care for her two children.

Further reading:

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Australia.

Categories: Aligned Planets

Colin Charles: Alan Knott Craig, Mxit & African mobile tech

Planet LCA 2009 - October 15, 2012 - 12:51

I know nothing about the African continent, having never stepped foot into it before (something I’m sure I will remedy within the next decade). I read a piece in the FT Weekend about Alan Knott Craig, Jr. (@alanknottcraig), an entrepreneur in South Africa that runs Mxit. Mxit is impressive: 750 million messages a day served, plus allowing 581 million mobile users in Africa to make electronic payments.

You have got to love Mxit’s mantra: help more Africans make more money.

This is a social entrepreneur at his best. I’ve already picked up his book titled: Mobinomics: Mxit and Africa’s Mobile Revolution, which I presume will be an interesting read.

He is also a workaholic with suggestions on how to improve his work-life balance with 3 simple rules:

  1. no working after 6pm
  2. no working on Sundays; and
  3. no travelling for more than seven consecutive nights

I just subscribed to his blog and followed him on Twitter and am totally eager to learn more about this amazing continent.

Related posts:

  1. My first Mobile Monday
  2. Notes from the Open Mobile Exchange
  3. MNP here; mobile content thoughts

Categories: Aligned Planets

Mary Gardiner: Sunday spam: porridge and honey

Planet LCA 2009 - October 14, 2012 - 08:59

What is cultural appropriation?

The problem isn’t that cultures intermingle, it’s the terms on which they do so and the part that plays in the power relations between cultures. The problem isn’t “taking” or “borrowing”, the problem is racism, imperialism, white supremacy, and colonialism. The problem is how elements of culture get taken up in disempowering, unequal ways that deny oppressed people autonomy and dignity. Cultural appropriation only occurs in the context of the domination of one society over another, otherwise known as imperialism. Cultural appropriation is an act of domination, which is distinct from ‘borrowing’, syncretism, hybrid cultures, the cultures of assimilated/integrated populations, and the reappropriation of dominant cultures by oppressed peoples.

Aircraft Carriers in Space

An article about naval metaphors in fictional space warfare. Sometimes I suspect that I like science fiction meta way more than I like science fiction.

“I’m not like the other girls.”

A quote I saw making the Tumblr rounds, which said, “I’m not like other girls!” It went on to avow wearing Converse instead of heels, preferring computer games to shopping, so on and so forth. When I saw it, about 41,000 girls had said they weren’t like “the others.”

Is Australia in Danger of Becoming Greece? Austerity and Blackmail Down Under

It is not enough to respond to this ongoing rhetoric about Australia’s supposed calamitous future by pointing out, as Ms Gillard correctly did, that these comparisons are ridiculous given the state of European periphery countries. Yet the ideological blackmail is strangely telling, precisely because the financial sector in the form of the troika (the International Monetary Fund, the European Commission and the European Central Bank) has held Greece’s politicians hostage, forcing a slashing of the government in exchange for “bail-out” loans.

The Start-to-Hate Review System

The concept is simple: Rate media based on how long it takes to encounter something bigoted. The longer it takes, the better the media.

An Investigation Into Xinjiang’s Growing Swarm of Great Gerbils

I am subscribed to two “long form” websites: the picks of Long Reads, which focuses on newer pieces, and the editor’s picks of Longform, which tend to skew a little older. Hence, this, from McSweeny’s in January 2005. I always like a piece that clearly ended up not being about what the original pitch was about. In this case, the writer wanted (or supposedly wanted, I guess) to investigate a gerbil plague, and ended up writing an article about gerbil social structures, text messaging on Chinese phone networks, and, several times, the Black Death. Which is how I ended up reading Wikipedia articles about pandemics the same night I was getting sick with the first illness I’ve had since I got out of hospital.

Mariana Trench Explosion

I think of Randall Munroe as a science writer who happens to be funded by merchandise sales from a comic. I don’t regularly look at the comic any more but I follow his blag and his What If? Answering your hypothetical questions with physics, every Tuesday writing more closely. This What If? is one of my favourites to date, although it’s hard to beat the first one. However, this one features an excursion into unpublished work by Freeman Dyson. SO HARD TO CHOOSE.

Do bicycle helmets reduce head injuries?

It’s impossible to follow Liam Hogan on Twitter without becoming interested in urban transport issues. At the moment the big conversation is helmet laws in Australia, which are arguably interfering with take-up of bike share schemes (if you’re going to have to get hold of a helmet, you don’t just jump on the bike, hence, scheme falls apart), although see Why is Brisbane CityCycle an unmitigated flop? for several other reasons that scheme may be failing.

Anyway, this one: A new study reports the rate of hospitalisations for cycling-related head injuries in NSW has fallen markedly and consistently since 1990. The authors say it’s due to helmets and infrastructure.

The drugs don’t work: a modern medical scandal and Ben Goldacre: ‘It’s appalling … like phone hacking or MPs’ expenses’

Reboxetine is a drug I have prescribed. Other drugs had done nothing for my patient, so we wanted to try something new. I’d read the trial data before I wrote the prescription, and found only well-designed, fair tests, with overwhelmingly positive results. Reboxetine was better than a placebo, and as good as any other antidepressant in head-to-head comparisons… In October 2010, a group of researchers was finally able to bring together all the data that had ever been collected on reboxetine, both from trials that were published and from those that had never appeared in academic papers. When all this trial data was put together, it produced a shocking picture. Seven trials had been conducted comparing reboxetine against a placebo. Only one, conducted in 254 patients, had a neat, positive result, and that one was published in an academic journal, for doctors and researchers to read. But six more trials were conducted, in almost 10 times as many patients. All of them showed that reboxetine was no better than a dummy sugar pill. None of these trials was published. I had no idea they existed.

Given that I favourited two separate articles about this, I’m going to buy the book. Now you know.

Going blind? DRM will dim your world

[I]t turned out I needed Adobe Digital Editions to ‘manage my content’… It tried, of course, to force me to give Adobe my email and other details for the ‘Adobe ID’ that it assured me I needed to get full functionality. I demurred… and was confronted by a user interface that was tiny white text on a black background. Unreadable. Options to change this? If they exist, I couldn’t find them.

Getting this far had taken me half an hour fighting my way through a nest of misery and frustration with broken eyes and a sinking heart. Along the way, I’d been bombarded by marketing messages telling me to “enjoy the experience” and “enjoy your book”.

Reader, I wept. Marketing departments, here’s a top tip: if your customer is reduced to actual, hot, stinging tears, you may wish to fine-tune your messaging.

5 Plans to Head Off the Apophis Killer Asteroid

Friday the 13th of April 2029 could be a very unlucky day for planet Earth. At 4:36 am Greenwich Mean Time, a 25-million-ton, 820-ft.-wide asteroid called 99942 Apophis will slice across the orbit of the moon and barrel toward Earth at more than 28,000 mph. The huge pockmarked rock, two-thirds the size of Devils Tower in Wyoming, will pack the energy of 65,000 Hiroshima bombs–enough to wipe out a small country or kick up an 800-ft. tsunami.

On this day, however, Apophis is not expected to live up to its namesake, the ancient Egyptian god of darkness and destruction. Scientists are 99.7 percent certain it will pass at a distance of 18,800 to 20,800 miles… Scientists calculate that if Apophis passes at a distance of exactly 18,893 miles, it will go through a “gravitational keyhole.” This small region in space–only about a half mile wide, or twice the diameter of the asteroid itself–is where Earth’s gravity would perturb Apophis in just the wrong way, causing it to enter an orbit seven-sixths as long as Earth’s. In other words, the planet will be squarely in the crosshairs for a potentially catastrophic asteroid impact precisely seven years later, on April 13, 2036.

It turns out that with current technology we might be able to move the asteroid prior to the (potential) 2029 entry into the gravitational keyhole, but if it did so we would be unlikely to perturb the orbit sufficiently after that point to avoid a civilisation-ended impact. So it’s the question of how many resources to spend on a low-probability but enormously catastrophic event.

Categories: Aligned Planets

Colin Charles: The chop space (digital loyalty cards) in Malaysia

Planet LCA 2009 - October 10, 2012 - 10:43

I love competition and free markets. I read about Pirq coming to Malaysia via the webcampkl group. An interesting thread is brewing.

In Malaysia, I see three players (not including Foursquare for merchants which some establishments use to give mayor discounts, every 5th check-in, etc.): 

  1. ChopChop is the pioneer in this space (bootstrapped around December 2011 by 3 passionate young entrepreneurs whom I’ve had the pleasure of meeting more than once). 
  2. Shortly thereafter Voucheres came along and they picked up a nifty RM700,000 from MyEG & MDEC (newsclip, crunchbase). They’re a startup with 4 founders (January 2012), claiming 10 employees, and they seem to be relaunching 10 months into it. 
  3. And the latest to the block? ChopInk (July 2012). Four young founders who were AllStars graduates (RM18,000 + mentorship for some 6-7% equity), which I’m told reliably is a pivot from a different unworkable idea. ChopInk has goals of a 1,000 merchants by year end and is supported by Cradle and possibly had some investment from Telekom Malaysia.

And today, you’ve got the fourth player: Pirq. Pirq’s take is different: you receive an immediate discount of 20-50% instantly. You don’t collect chops for later redemption. Pirq is a US-based company flush with cash – currently USD$3.2 million has been raised (yes, thats USD not Ringgit). Their first expansion country: Malaysia, then Singapore. Pirq is like collecting chops meets Groupon (20-50% discounts on a bill last I checked at most restaurants is unsustainable). 

The grapevine tells me that Pirq has four sales people on-board. From an execution perspective, I love how they focus on areas. My biggest problem with these digital loyalty card applications is that I generally never visit any of their merchants! From a tech perspective, Pirq needs work.

I see Pirq as competition with group buying sites, which is definitely seeing fatigue (in Singapore they’re dwindling; in Malaysia?). The verdict is still out there how digital loyalty is going to be managed between ChopChop, ChopInk, Voucheres. Maybe well-funded Singaporean Perx might arrive eventually.

As a consumer, while I may not have to collect loyalty cards in my wallet any longer, I’m going to be collecting smartphone apps. Good thing you have folders on iOS :)

Who’s going to win? The people that make the better product & with better execution. Not just for the consumer (location based alerts, geo-fencing, etc) but for the merchants as well (smart ad posting, etc.). 

In another post, we’ll talk about money. Foreign money is rolling into companies coming into Malaysia (Rocket Internet, now Pirq), mainly because the USD or Euro goes further in Ringgit Malaysia land. Most of the discussion at webcampkl is focused on this.

Me? I’m naturally rooting for the bootstrapped entrepreneurs – that’s ChopChop.

Related posts:

  1. Tax incentives for angel investors in Malaysia
  2. Silly Malaysia – race cards pulled out, what happens when oil ringgit runs out?
  3. zalora malaysia: some quick thoughts

Categories: Aligned Planets

Tim Connors: Terry Mulder, minister for transport or just stationary cars?

Planet LCA 2009 - October 6, 2012 - 12:09
This wasn't published in The Age presumably not because it wasn't brilliant, but because I was 26 words over the 200 word limit:

Why does State Transport Minister Terry Mulder think that building an East-West freeway link will help solve problems similar to today's -- the likes of which happens roughly once every 13 years? To be most useful (but still with only a return on investment of 0.7), there can't be any off-ramps into the city, otherwise the congestion will just be transferred to city roads instead of Hoddle Street. It's pure an East-West link. Today, the blockages at the tunnel were stopping people getting into the city. How does as East West link solve problems like today's (or indeed any other problem that can't be solved by improving railway freight links instead)?

Meanwhile, we've got a train system that actually suffered a decline in patronage this year because the train timetable change last year made it so unattractive to catch a train anywhere, because formerly simple trips now involve poorly timed unnecessary changes of train and platform. I recently found it quicker to walk home 2 stations from the closest premium station because Metro Trains were so disorganised and seemed to have misplaced their connecting train (station staff certainly had no idea what was going on).

Made me glad to be on my bike next to the freeway today as I rode past all the stationary traffic. The only reliable way to get to and from work.

Categories: Aligned Planets

Keith Packard: fd-passing

Planet LCA 2009 - October 6, 2012 - 07:30
FD passing for DRI.Next

Using the DMA-BUF interfaces to pass DRI objects between the client and server, as discussed in my previous blog posting on DRI-Next, requires that we successfully pass file descriptors over the X protocol socket.

Rumor has it that this has been tried and found to be difficult, and so I decided to do a bit of experimentation to see how this could be made to work within the existing X implementation.

(All of the examples shown here are licensed under the GPL, version 2 and are available from git://

Basics of FD passing

The kernel internals that support FD passing are actually quite simple — POSIX already require that two processes be able to share the same underlying reference to a file because of the semantics of the fork(2) call. Adding some ability to share arbitrary file descriptors between two processes then is far more about how you ask the kernel than the actual file descriptor sharing operation.

In Linux, file descriptors can be passed through local network sockets. The sender constructs a mystic-looking sendmsg(2) call, placing the file descriptor in the control field of that operation. The kernel pulls the file descriptor out of the control field, allocates a file descriptor in the target process which references the same file object and then sticks the file descriptor in a queue for the receiving process to fetch.

The receiver then constructs a matching call to recvmsg that provides a place for the kernel to stick the new file descriptor.

A helper API for testing

I first write a stand-alone program that created a socketpair, forked and then passed an fd from the parent to the child. Once that was working, I decided that some short helper functions would make further testing a whole lot easier.

Here’s a function that writes some data and an optional file descriptor:

ssize_t sock_fd_write(int sock, void *buf, ssize_t buflen, int fd) { ssize_t size; struct msghdr msg; struct iovec iov; union { struct cmsghdr cmsghdr; char control[CMSG_SPACE(sizeof (int))]; } cmsgu; struct cmsghdr *cmsg; iov.iov_base = buf; iov.iov_len = buflen; msg.msg_name = NULL; msg.msg_namelen = 0; msg.msg_iov = &iov; msg.msg_iovlen = 1; if (fd != -1) { msg.msg_control = cmsgu.control; msg.msg_controllen = sizeof(cmsgu.control); cmsg = CMSG_FIRSTHDR(&msg); cmsg->cmsg_len = CMSG_LEN(sizeof (int)); cmsg->cmsg_level = SOL_SOCKET; cmsg->cmsg_type = SCM_RIGHTS; printf ("passing fd %d\n", fd); *((int *) CMSG_DATA(cmsg)) = fd; } else { msg.msg_control = NULL; msg.msg_controllen = 0; printf ("not passing fd\n"); } size = sendmsg(sock, &msg, 0); if (size < 0) perror ("sendmsg"); return size; }

And here’s the matching receiver function:

ssize_t sock_fd_read(int sock, void *buf, ssize_t bufsize, int *fd) { ssize_t size; if (fd) { struct msghdr msg; struct iovec iov; union { struct cmsghdr cmsghdr; char control[CMSG_SPACE(sizeof (int))]; } cmsgu; struct cmsghdr *cmsg; iov.iov_base = buf; iov.iov_len = bufsize; msg.msg_name = NULL; msg.msg_namelen = 0; msg.msg_iov = &iov; msg.msg_iovlen = 1; msg.msg_control = cmsgu.control; msg.msg_controllen = sizeof(cmsgu.control); size = recvmsg (sock, &msg, 0); if (size < 0) { perror ("recvmsg"); exit(1); } cmsg = CMSG_FIRSTHDR(&msg); if (cmsg && cmsg->cmsg_len == CMSG_LEN(sizeof(int))) { if (cmsg->cmsg_level != SOL_SOCKET) { fprintf (stderr, "invalid cmsg_level %d\n", cmsg->cmsg_level); exit(1); } if (cmsg->cmsg_type != SCM_RIGHTS) { fprintf (stderr, "invalid cmsg_type %d\n", cmsg->cmsg_type); exit(1); } *fd = *((int *) CMSG_DATA(cmsg)); printf ("received fd %d\n", *fd); } else *fd = -1; } else { size = read (sock, buf, bufsize); if (size < 0) { perror("read"); exit(1); } } return size; }

With these two functions, I rewrote the simple example as follows:

void child(int sock) { int fd; char buf[16]; ssize_t size; sleep(1); for (;;) { size = sock_fd_read(sock, buf, sizeof(buf), &fd); if (size <= 0) break; printf ("read %d\n", size); if (fd != -1) { write(fd, "hello, world\n", 13); close(fd); } } } void parent(int sock) { ssize_t size; int i; int fd; fd = 1; size = sock_fd_write(sock, "1", 1, 1); printf ("wrote %d\n", size); } int main(int argc, char **argv) { int sv[2]; int pid; if (socketpair(AF_LOCAL, SOCK_STREAM, 0, sv) < 0) { perror("socketpair"); exit(1); } switch ((pid = fork())) { case 0: close(sv[0]); child(sv[1]); break; case -1: perror("fork"); exit(1); default: close(sv[1]); parent(sv[0]); break; } return 0; } Experimenting with multiple writes

I wanted to know what would happen if multiple writes were made, some with file descriptors and some without. So I changed the simple example parent function to look like:

void parent(int sock) { ssize_t size; int i; int fd; fd = 1; size = sock_fd_write(sock, "1", 1, -1); printf ("wrote %d without fd\n", size); size = sock_fd_write(sock, "1", 1, 1); printf ("wrote %d with fd\n", size); size = sock_fd_write(sock, "1", 1, -1); printf ("wrote %d without fd\n", size); }

When run, this demonstrates that the reader gets two bytes in the first read along with a file descriptor followed by one byte in a second read, without a file descriptor. This demonstrates that a file descriptor message forms a barrier within the socket; multiple messages will be merged together, but not past a message containing a file descriptor.

Reading without accepting a file descriptor

What happens when the reader isn’t expecting a file descriptor? Does it just get lost? Does the reader not get the message until it asks for the file descriptor? What about the boundary issue described above?

Here’s my test case:

void child(int sock) { int fd; char buf[16]; ssize_t size; sleep(1); size = sock_fd_read(sock, buf, sizeof(buf), NULL); if (size <= 0) return; printf ("read %d\n", size); size = sock_fd_read(sock, buf, sizeof(buf), &fd); if (size <= 0) return; printf ("read %d\n", size); if (fd != -1) { write(fd, "hello, world\n", 13); close(fd); } } void parent(int sock) { ssize_t size; int i; int fd; fd = 1; size = sock_fd_write(sock, "1", 1, 1); printf ("wrote %d without fd\n", size); size = sock_fd_write(sock, "1", 1, 2); printf ("wrote %d with fd\n", size); }

This shows that the first passed file descriptor is picked up by the first sockfdread call, but the file descriptor is closed. The second file descriptor passed is picked up by the second sockfdread call.

Zero-length writes

Can a file descriptor be passed without sending any data?

void parent(int sock) { ssize_t size; int i; int fd; fd = 1; size = sock_fd_write(sock, "1", 1, -1); printf ("wrote %d without fd\n", size); size = sock_fd_write(sock, NULL, 0, 1); printf ("wrote %d with fd\n", size); size = sock_fd_write(sock, "1", 1, -1); printf ("wrote %d without fd\n", size); }

And the answer is clearly “no” — the file descriptor is not passed when no data are included in the write.

A summary of results
  1. read and recvmsg don’t merge data across a file descriptor message boundary.

  2. failing to accept an fd in the receiver results in the fd being closed by the kernel.

  3. a file descriptor must be accompanied by some data.

Make X pass file descriptors

I’d like to get X to pass a file descriptor without completely rewriting the internals of both the library and the X server. Ideally, without making any changes to the existing code paths for regular request processing at all.

On the sending side, this seems pretty straightforward — we just need to get the X connection file descriptor and call sendmsg directly, passing the desired file descriptor along. In XCB, this could be done by using the xcbtakesocket interface to temporarily hijack the protocol as Xlib does.

It’s the receiving side where things are messier. Because a bare read will discard any delivered file descriptor, we must make sure to use recvmsg whenever we want to actually capture the file descriptor.

Kludge X server fd receiving

Because a passed fd creates a barrier in the bytestream, when the X server reads requests from a client, the read will stop sending data after the message with the file descriptor is consumed.

Of course, this process consumes the passed file descriptor, and if that call isn’t made with recvmsg set up to receive it, the fd will be lost.

As a simple kludge, if we pass a meaningless fd with the X request and then the ‘real’ fd with a following XNoOperation request, the existing request reading code will get the request, discard the meaningless fd and then stop reading at that point due to the barrier. Once into the request processing code, recvmsg can be called to get the real file descriptor and the associated XNoOperation request.

I wrote a test for this that demonstrates how this works:

static void child(int sock) { uint8_t xreq[1024]; uint8_t xnop[4]; uint8_t req; int i, reqlen; ssize_t size, fdsize; int fd = -1, *fdp; int j; sleep (1); for (j = 0;; j++) { size = sock_fd_read(sock, xreq, sizeof (xreq), NULL); printf ("got %d\n", size); if (size == 0) break; i = 0; while (i < size) { req = xreq[i]; reqlen = xreq[i+1]; i += reqlen; switch (req) { case 0: break; case 1: if (i != size) { fprintf (stderr, "Got fd req, but not at end of input %d < %d\n", i, size); } fdsize = sock_fd_read(sock, xnop, sizeof (xnop), &fd); if (fd == -1) { fprintf (stderr, "no fd received\n"); } else { FILE *f = fdopen (fd, "w"); fprintf(f, "hello %d\n", j); fflush(f); fclose(f); close(fd); fd = -1; } break; case 2: fprintf (stderr, "Unexpected FD passing req\n"); break; } } } } int tmp_file(int j) { char name[64]; sprintf (name, "tmp-file-%d", j); return creat(name, 0666); } static void parent(int sock) { uint8_t xreq[32]; uint8_t xnop[4]; int i, j; int fd; for (j = 0; j < 4; j++) { /* Write a bunch of regular requests */ for (i = 0; i < 8; i++) { xreq[0] = 0; xreq[1] = sizeof (xreq); sock_fd_write(sock, xreq, sizeof (xreq), -1); } /* Write our 'pass an fd' request with a 'useless' FD to block the receiver */ xreq[0] = 1; xreq[1] = sizeof(xreq); sock_fd_write(sock, xreq, sizeof (xreq), 1); /* Pass an fd */ xnop[0] = 2; xnop[1] = sizeof (xnop); fd = tmp_file(j); sock_fd_write(sock, xnop, sizeof (xnop), fd); close(fd); } } Fixing XCB to receive file descriptors

Multiple threads may be trying to get replies and events back from the X server at the same time, which means the kludge of having the real fd follow the message will likely lead to the wrong thread getting the file descriptor.

Instead, I suspect the best plan will be to fix XCB to internally capture passed file descriptors and save them with the associated reply. Because the file descriptor message will form a barrier in the read stream, xcb can associate any received file descriptor with the last reply in the read data. The X server would then send the reply with an explicit sendmsg call to pass both reply and file descriptor together.

Next steps

The next thing to do is code up a simple fd passing extension and try to get it working, passing descriptors back and forth to the X server. Once that works, design of the rest of the DRM-Next extension should be pretty straightforward.

Categories: Aligned Planets

Colin Charles: The link between Yahoo!, Microsoft, Facebook, Nokia

Planet LCA 2009 - October 4, 2012 - 11:43

Written 23 July 2012, but for some reason it never got posted. Better late than never I guess.

I tweeted (17 July 2012, 4:40am UTC+8): 

There’s an interesting link between Yahoo!, Microsoft, Bing, Facebook and Nokia. The bigger picture is competition against Google, Apple

This was literally moments after the news broke that Marissa Mayer resigned from Google to become the CEO of Yahoo!. I thought I’d expand on this link that I see.

Search is today not something that Yahoo! cares about. Its served by Bing from Microsoft. Bing is also the default on Windows Phone, the operating system that Nokia has taken a bet on (when in the USA, I use a Lumia Windows Phone and cannot complain). Search on Facebook is also powered by Bing thanks to a deal that Microsoft has with Facebook. Bing is a strong contender to Google’s search, and this space is clearly still getting investment (see how DuckDuckGo recently got VC funding too).

Yahoo! has mail that is very popular (it might still be the most popular out there). Microsoft has Hotmail. Facebook has “Facebook messages”. Nokia canned Ovi mail services. Yahoo!, Microsoft Messenger and Facebook Messenger also has instant messenger (IM) capabilities. Imagine a day when they all interconnect? It would be a straight fight against Google Chat.

Picasa is Google’s photo sharing site. Today the stream might be Google+. Yahoo! still has Flickr which is the Picasa equivalent, and for streaming? Imagine if there was a quick link to Facebook. Nokia can build in sharing to Flickr and Facebook quickly from their phones (they already have been doing this from time-to-time between phone releases including their MeeGo stint).

Videos seem to be missing from this big picture. Google has YouTube, and the rest of them have nothing with the exception of Facebook.

Maps? Nokia has got great mapping technology loaded on the Windows phone. It can supply this quite easily to everyone.

I haven’t once mentioned Apple yet. They use other search engines (and maybe the longer term strategy is something like what the Dolphin browser does: use Siri to search multiple search engines and aggregate the results so the user has no idea what search engine is being used). They have their own messenger service in iMessages. They have their own photo & video storage site – the iCloud. For maps, they are using OpenStreetMaps after having ditched Google Maps. I see Apple building their own ecosystem and going it alone.

What about developer appeal? I see many a developer hacking on a Mac OS X laptop or a Linux laptop. With the Apple ecosystem, it is obvious to develop on OSX. With the Google Android ecosystem and the rest of their toolkit, its clear you can be OS-agnostic (they support Mac, Linux, Windows). With the Microsoft/Nokia ecosystem? It seems like you need a Windows box, and that automatically turns me away quite quickly (though upcoming HTML5/CSS/JavaScript will allow more development on this platform, in an OS-agnostic sense). Facebook is OS-agnostic too.

It is an exciting time ahead. All of this is great for consumers! Ecosystems are a building and it is awesome to see alliances being built

Related posts:

  1. HTC, Android, Facebook
  2. Microsoft blackouts… Software Freedom
  3. A letter to the folk at Yahoo!

Categories: Aligned Planets

LCA2009 News: LCA2013 Earlybird Registrations Open

Planet LCA 2009 - October 2, 2012 - 15:45
Let the Celebrations Begin!, one of the largest open source conferences in the southern hemisphere, has now opened registrations. For a strictly limited time, discounted 'early bird' tickets are available through the conference website at

The 2013 conference builds on a long tradition of sharing technical know-how between seasoned open source gurus and newcomers to the community. Since its inception in 1999, the conference has moved around Australia and New Zealand, most recently to Ballarat, Victoria, and Brisbane, Queensland. This year, the conference is in Canberra in celebration of our national capital’s centenary year. The conference was last hosted in Canberra in 2005, and it has grown significantly since then, bringing some unique challenges to the organising team.

In true open source style, the conference is run entirely by volunteers, who are often drawn from a local Linux user group. This year is no exception, with the core organising team all being long standing members of the Canberra Linux Users Group (CLUG), which hosts meetings at the Australian National University (ANU) once a month. CLUG and the ANU have been major supporters of for many years running, and are proud to be involved again this year.

Many of the original 2005 organisers have returned for the 2013 effort, including Michael Still, who has stepped into the shoes of Conference Director (and is affectionately referred to as “The Grand Catamaran” for reasons known only to the core organising team). Michael says that, while the role of Conference Director is time consuming and sometimes stressful, it also can also be very rewarding: "the opportunity to work with so many talented people, to rub shoulders with the open source elite, and of course to develop skills in new areas, is very valuable and also a whole lot of fun".

The current organising committee has been working together since mid-2011 in order to bring the 2013 conference together. There's a lot to do, with the conference running over six days, offering over 100 presentations (including four keynotes and twelve miniconferences), four formal social events, delegate accommodation, hundreds of giveaways, and moving over 500 potential delegates into and around the city. The group is as diverse as the tasks they need to complete, though, with highly experienced developers, systems administrators, engineers, and technical communicators amongst the team.

Perhaps the most important part of any conference is the speaker lineup. has broken with tradition and announced their conference programme much earlier than usual. Featured on the programme this year are open source luminaries such as Jonathan Oxer, Noirin Plunkett, and local developer Andrew Tridgell. The conference organisers, together with the papers committee, have gone to great effort to ensure that the programme is stocked with deep technical content. The focus this year is on what's coming on the technical horizon, from the latest developments in the the Linux kernel, to working with remote clusters and cloud technology. Still to be announced, however, are the four keynote speakers. Traditionally, conference keynotes are big names in the open source space, with previous years boasting such respected speakers as Jacob Applebaum and Vint Cerf.

The social events are also an important part of the experience for delegates, with the main conference dinner, called the 'penguin dinner', the highlight of the week. This year, delegates are being treated to a relaxed evening on Mount Stromlo, where they will be able to delight in some of the best views the city has to offer, and enjoy a relaxed 'backyard barbecue'-style dinner. The team are also promising some other events for the evening, just in case the view isn't quite exciting enough for you. The other highly anticipated event is the Professional Delegates' Networking Session (PDNS), which for the first time is being held as a breakfast, located in the breathtaking Gandel Hall at the National Gallery of Australia.

As an open source event, the conference is largely reliant on their sponsors. is overseen and managed by Linux Australia, who use the conference as their primary incubator for open source development throughout Australia and New Zealand. This year, is also supported by IBM, HP, Anchor Systems, Defence Signals Directorate, and Linux Magazine. Without the generous help of these partner organisations, there would be no

Canberra is expected to be in full party mode during 2013, with the ACT Government spending over $30 million to ensure that the city celebrates its 100th birthday in serious style, and 2013 will be no exception. Head on over to now and grab your discounted early bird ticket quickly while they last. Then go and put your party shoes in your suitcase, because you're going to need them.

About showcases the best of open source and community-driven software and hardware, and it’s coming to the Australian National University from 28 January to 2 February, 2013. The conference provides a great opportunity for open source developers, users, hackers, and makers to share their ideas and further improve their projects. More information and tickets are available from

Categories: Aligned Planets

Keith Packard: DRI-Next

Planet LCA 2009 - September 29, 2012 - 06:08
Thoughts about DRI.Next

On the way to the X Developer’s Conference in Nuremberg, Eric and I chatted about how the DRI2 extension wasn’t really doing what we wanted. We came up with some fairly rough ideas and even held an informal “presentation” about it.

We didn’t have slides that day, having come up with the content for the presentation in the hours just before the conference started. This article is my attempt to capture both that discussion and further conversations held over roast pork dinners that week.

A brief overview of DRI2

Here’s a list of the three things that DRI2 currently offers.

Application authentication.

The current kernel DRM authentication mechanism restricts access to the GPU to applications connected to the DRM master. DRI2 implements this by having the application request the DRM cookie from the X server which can then be passed to the kernel to gain access to the device.

This is fairly important because once given access to the GPU, an application can access any flink’d global buffers in the system. Given that the application sends screen data to the X server using flink’d buffers, that means all screen data is visible to any GPU-accessing application. This bypasses any GPU hardware access controls.

Allocating buffers.

DRI2 defines a set of ‘attachment points’ for buffers which can be associated with an X drawable. An application needing a specific set of buffers for a particular rendering operation makes a request of the X server which allocates the buffers and passes back their flink names.

The server automatically allocates new buffers when window sizes change, sending an event to the application so that it knows to request the new buffers at some point in the future.

Presenting data to the user.

The original DRI2 protocol defined only the DRI2CopyRegion request which copied data between the allocated buffers. SwapBuffers was implemented by simply copy data from the back buffer to the front buffer. This didn’t provide any explicit control over frame synchronization, so a new request, DRI2SwapBuffers, was added to expose controls for that. This new request only deals with the front and back buffers, and either copies from back to front or exchanges those two buffers.

Along with DRI2SwapBuffers, there are new requests that wait for various frame counters and expose those to GL applications through the OMLsynccontrol extension

What’s wrong with DRI2?

DRI2 fixed a lot of the problems present with the original DRI extension, and made reliable 3D graphics on the Linux desktop possible. However, in the four years since it was designed, we’ve learned a lot, and the graphics environment has become more complex. Here’s a short list of some DRI2 issues that we’d like to see fixed.

  • InvalidateBuffers events. When the X window size changes, the buffers created by the X server for rendering must change size to match. The problem is that the client is presumably drawing to the old buffers when the new ones are allocated. Delivering an event to the client is supposed to make it possible for the client to keep up, but the reality is that the event is delivered at some random time to some random thread within the application. This leads to general confusion within the application, and often results in a damaged frame on the screen. Fortunately, applications tend to draw their contents often, so the damaged frame only appears briefly.

  • No information about new back buffer contents. When a buffer swap happens and the client learns about the new back buffer, the back buffer contents are always undefined. For most applications, this isn’t a big deal as they’re going to draw the whole window. However, compositing managers really want to reduce rendering by only repainting small damaged areas of the window. Knowing what previous frame contents are present in the back buffer allows the compositing manager to repaint just the affected area.

  • Un-purgable stale buffers. Between the X server finishing with a buffer and the client picking it up for a future frame, we don’t need to save the buffer contents and should mark the buffer as purgable. With the current DRI2 protocols, this can’t be done, which leaves all of those buffers hanging around in memory.

  • Driver-specific buffers. The DRI2 buffer handles are device specific, and so we can’t use buffers from other devices on the screen. External video encoders/cameras/encoders can’t be used with the DRI2 extension.

  • GEM flink has lots of issues. The flink names are global, allowing anyone with access to the device to access the flink data contents. There is also no reference to the underlying object, so the X server and client must carefully hold references to GEM objects during various operations.

Proposed changes for DRI.Next

Given the three basic DRI2 operations (authentication, allocation, presentation), how can those be improved?

Eliminate DRI/DRM magic-cookie based authentication

Kristian Høgsberg, Martin Peres, Timothée Ravier & Daniel Vetter gave a talk on DRM2 authentication at XDC this year that outlined the problems with the current DRM access control model and proposed some fairly simple solutions, including using separate device nodes—one for access to the GPU execution environment and a separate, more tightly controlled one, for access to the display engine.

Combining that with the elimination of flink for communicating data between applications and there isn’t a need for the current magic-cookie based authentication mechanism; simple file permissions should suffice to control access to the GPU.

Of course, this ignores the whole memory protection issue when running on a GPU that doesn’t provide access control, but we already have that problem today, and this doesn’t change that, other than to eliminate the global uncontrolled flink namespace.

Allocate all buffers in the application

DRI2 does buffer allocation in the X server. This ensures that that multiple (presumably cooperating) applications drawing to the same window will see the same buffers, as is required by the GLX extension. We suspected that this wasn’t all that necessary, and it turns out to have been broken several years ago. This is the traditional way in X to phase out undesirable code, and provides an excellent opportunity to revisit the original design.

Doing buffer allocations within the client has several benefits:

  • No longer need DRI2 additions to manage new GL buffers. Adding HiZ to the intel driver required new DRI2 code in the X server, even though X wasn’t doing anything with those buffers at all.

  • Eliminate some X round trips currently required for GL buffer allocation.

  • Knowing what’s in each buffer. Because the client allocates each buffer, it can track the contents of them.

  • Size tracking is trivial. The application sends the GL the of the viewport, and the union of all viewports should be the same as the size of the window (or there will be undefined contents on the screen). The driver can use the viewport information to size the buffers and ensure that every frame on the screen is complete.

Present buffers through DMA-buf

The new DMA-buf infrastructure provides a cross-driver/cross-process mechanism for sharing blobs of data. DMA-buf provides a way to take a chunk of memory used by one driver and pass it to another. It also allows applications to create file descriptors that reference these objects.

For our purposes, it’s the file descriptor which is immediately useful. This provides a reliable and secure way to pass a reference from an underlying graphics buffer from the client to the X server by sending the file descriptor over the local X socket.

An additional benefit is that we get automatic integration of data from other devices in the system, like video decoders or non-primary GPUs. The ‘Prime’ support added in DRI version 2.8 hacks around this by sticking a driver identifier in the driverType value.

Once the buffer is available to the X server, we can create a request much like the current DRI2SwapBuffers request, except instead of implicitly naming the back and front buffers, we can pass an arbitrary buffer and have those contents copied or swapped to the drawable.

We also need a way to copy a region into the drawable. I don’t know if that needs the same level of swap control, but it seems like it would be nice. Perhaps the new SwapBuffers request could take a region and offset as well, copying data when swapping isn’t possible.

Managing buffer allocations

One trivial way to use this new buffer allocation mechanism would be to have applications allocate a buffer, pass it to the X server and then simply drop their reference to it. The X server would keep a reference until the buffer was no longer in use, at which point the buffer memory would be reclaimed.

However, this would eliminate a key optimization in current drivers— the ability to re-use buffers instead of freeing and allocating new ones. Re-using buffers takes advantage of the work necessary to setup the buffer, including constructing page tables, allocating GPU memory space and flushing caches.

Notifying the application of idle buffers

Once the X server is finished using a buffer, it needs to notify the application so that the buffer can be re-used. We could send these notifications in X events, but that ends up in the twisty mess of X client event handling which has already caused so much pain with Invalidate events. The obvious alternative is to send them back in a reply. That nicely controls where the data are delivered, but causes the application to block waiting for the X server to send the reply.

Fortunately, applications already want to block when swapping buffers so that they get throttled to the swap buffers rate. That is currently done by having them wait for the DRI2SwapBuffers reply. This provides a nice place to stick the idle buffer data. We can simply list buffers which have become idle since the last SwapBuffers reply was delivered.

Releasing buffer memory

Applications which update only infrequently end up with a back buffer allocated after their last frame which can’t be freed by the system. The fix for this is to mark the buffer purgable, but that can only be done after all users of the buffer are finished with it.

With this new buffer management model, the application effectively passes ownership of its buffers to the X server, and the X server knows when all use of the buffer are finished. It could mark buffers as purgable at that point. When the buffer was sent back in the SwapBuffers reply, the application would be able to ask the kernel to mark it un-purgable again.

A new extension? Or just a new DRI2 version?

If we eliminate the authentication model and replace the buffer allocation and presentation interfaces, what of the existing DRI2 protocol remains useful? The only remaining bits are the other synchronization requests: DRI2GetMSC, DRI2WaitMSC, DRI2WaitSBC and DRI2SwapInterval.

Given this, does it make more sense to leave DRI2 as it is and plan on deprecating, and eventually eliminating, it?

Doing so would place a support burden on existing applications, as they’d need to have code to use the right extension for the common requests. They’ll already need to support two separate buffer management versions though, so perhaps this burden isn’t that onerous?

Categories: Aligned Planets
Syndicate content