Research, Technology, and Teamwork blog by Susie Wee : technology

From cameraman to camerawoman in NIR imaging research

susie.wee — Sat, 14 Nov 2009 21:49:00 GMT

In the world of image processing research, it is important to have standard test images so people can compare their results. The cameraman image is a test image that has been used for decades. It can be found in many image processing textbooks and homework...(read more)

Social cloud technology in the fabric of the Internet

susie.wee — Sat, 15 Aug 2009 21:52:00 GMT

A shell for Twitter Remember the good old days when you had a terminal screen and you typed ls , cd , and man ? And, if you were a little more advanced, you might have used pushd , popd , cat , head , and tail . Well, there is a very alpha project called...(read more)

Back from sabbatical: Blogging is back, but comments and subscribers are not

susie.wee — Fri, 14 Aug 2009 07:49:00 GMT

I took a sabbatical from blogging for the last 1.5 years. It was not because I was making a statement. It was not because I switched to a flashier tool . It was just because I took a job that was not very conducive to blogging. In essence, I was immersed...(read more)

A cute workshop

susie.wee — Sat, 05 Jan 2008 10:11:00 GMT

I came across an announcement for the following workshop: Workshop on Designing Cute Interactive Media in conjunction with the ACM conference on Designing Interactive Systems.

ACM is the premier professional research society for computer scientists. I think it is quite a statement that the broader research community is recognizing design, experience, and human emotion as bona fide research topics. Cuteness is being recognized as research by the research community!

In my mind, user adoption is the ultimate indicator of a technology's success, and adoption is driven by having a great user experience. The research discussed in workshops like these will help us understand and eventually formalize the coupling of experience and technology. Understanding how to provoke human emotions like cuteness will help identify new research directions and drive technology adoption.

Congratulations to the researchers who were pushing these ideas in their work before it reached broader acceptance! For example, 2007 was the 25th anniversary of CHI. Clearly your efforts are paying off!

What do you think? Is studying "cuteness" research?

Feel free to leave a URL with your comments.

The Life of a Packet for Mobile & Media Experiences (Packet Video 2007 Keynote)

susie.wee — Tue, 13 Nov 2007 01:02:00 GMT

I gave a keynote talk to kickstart the Packet Video 2007 Workshop in Lausanne, Switzerland. The audience was great, and the talk seemed to generate lots of discussion during the Q&A and for the remainder for the workshop. Here's a recap.

Abstract

Mobile & media experiences connect people with each other, with information, and with their environment. Media is increasingly being delivered in packets over networks. This raises a number of questions for today's networks:

How can we transport media packets?

How can we adapt media packets for diverse clients?

How can we protect media packets?
A number of emerging applications will impact future directions for packet networks. We also discuss the following questions:

What impact do globally distributed, immersive media environments have on media packet delivery systems?

What role does context play in next-generation mobile media experiences?
We consider these questions from the perspective of a user and the perspective of a packet.

Coupling experience and technology

I began by stressing the importance of coupling experience and technology. Rather than developing technology in a box, it is important to first consider the desired user experience and then develop the technologies that impact it. The most important factor for deciding whether a technology gets transferred to product is not how good the technology is, but rather how it impacts the user experience. I have been passionate about this theme for quite some time, and as time passes my passion for this only grows stronger.

The rest of my talk cycled between the following experiences and technologies.

Mobile & Media Experiences

Experience #1: Mobile, Diverse, Interactive: Diverse mobile video clients, desktop video, living room video
Experience #2: Immersive, Conversational, Worldwide: Halo collaboration experience, Panoply immersive gaming experience
Experience #3: Pervasive, Personalized, Context-aware: Mediascapes context-aware multimedia experience

Packet Technologies

Packet labeling & metadata
Transcoding & Processing in the network
Scalable Streaming
Secure Scalable Streaming
Multiple Distortion Measures
Public & private domains
Sensing context in the network

The first five technologies were discussed in the context of Experience #1. The last two were discussed with Experience #2 and #3.

Experience #1: Mobile, Diverse, Interactive

Packet labeling & metadata: The main point is that we live in a distributed networked world where media packets will traverse distributed network elements with multiple owners and administrative domains and be processed by devices and equipment made by different manufacturers. In this highly distributed world, one important thing that we can do is smartly label our packets in hopes that over time the smart network elements along the way will use these labels to improve the overall quality of the user experience. The key design principle is to design packet labels that are 1) specific enough to be useful and 2) general enough to be understood.

Example packet labels and metadata include:

Importance: Distortion values
Time requirements: Time stamps
Content type: Video, audio, text, data
Scalability: Is it truncatable?
Media attributes: spatial region, resolution, color; audio channel
Dropability: Can it be dropped? e.g., Drop video for audio-only session.
Processibility: Is it transcodable? Can it be processed?
Security: What are the rights and privacy implications of the media?

The research challenges are designing and standardizing the labels with the design principle above, and then developing algorithms that use these labels for delivering improved mobile media experiences. These algorithms should be evaluated for their performance gains with respect to the label overhead.

Transcoding & Processing in the network

I discussed the experience of delivering media to and from users over any network and on any device. This motivates the technology of performing transcoding operations in the network. In 3G networks, the streaming, recording, and transcoding capabilities can be performed by the IMS Multimedia Resource Function (MRF), which serves and receives the media packets to and from the handsets. Dynamic transcoding can be used to adapt the video for the target client device (e.g., to lower the resolution) and for the network (e.g., to seamlessly handoff media between 3G and 2.5G networks during a mobile media session).

The research challenge that lies ahead is designing and developing transcoding algorithms in a manner that is computationally efficient so that a single transcoding node (e.g., IMS MRF) can process many streams at once to serve multiple clients at one time.

Scalable Streaming

This brings us to a technology called scalable streaming that makes transcoding much more efficient by leveraging scalable coding methods. In essence, if scalable coding methods are used, then we can form scalable packets that pack scalable data, for example low, medium, and high resolution data, into the packet in a manner that allows it to be transcoded by simply truncating the packet. Furthermore, the scalable media packets can have packet labels that contain image metadata and truncation points that can be used by a scalable packet transcoder. The scalable packet transcoder is quite simple- it performs transcoding by simply reading the packet label and then truncating the packet as needed.

Research opportunities arise if the packet labels contain the distortion value of the particular media packet. If distortion values are included in the label, then they can be used as hints for rate-distortion optimized streaming algorithms and rate-distortion optimized transcoding algorithms to improve the quality of the user experience.

Secure Scalable Streaming

Another desired experience includes serving diverse clients while having end-to-end security. End-to-end security means that the media is protected in a manner that only allows the sender and allowed receivers to access the media, while delivering, storing, and transcoding the media packet over the network in a way that does not require decryption. It turns out that this can be achieved by using the same method as scalable streaming, where scalable packets are formed by leveraging scalable coding, and then coupling the packet formation with the encryption process. Specifically, encryption is applied to the packet in a manner that allows the packet transcoding operation to still occur by simple packet truncation. This can also leverage secure scalable image coding standards such as the newly created JPSEC standard for security of JPEG-2000 imagery.

Secure Scalable Streaming was published in ICASSP 2001 by Susie Wee and John Apostolopoulos.

Multiple Distortion Measures

I then described a new technology area that we are studying called Multiple Distortion Measures (MDM). This begins with the following observation: Consider a set of scalable media packets. Generally speaking, the best ordering of the packets is determined by the profit-to-size ratio (or distortion-to-size ratio, in tech terms, delta d over delta r). Surprisingly, we observed that the best ordering for low resolution display is NOT equal to the best ordering for high resolution display. The question that arises is how different are they?

I showed a graph from our ICASSP 2007 paper that shows the PSNR vs. Rate plot for the low resolution reconstructed image with packets ordered in the low-res optimal order and with packets in the high-res optimal order. It turns out that there are differences in performance of up to 4 dB. The graph aso showed the PSNR vs. Rate plot for the high resolution reconstructed image with packets ordered in the high-res optimal order and the low-res optimal order. It turns out that these can have differences of over 1 dB.

This raised a lot of interest from the crowd. I think we'll have lots of people researching MDMs in the years ahead.

This raises the idea of labeling scalable media packets with multiple distortion measures, specifically, with the distortion value of the packet with respect to the low resolution image, the medium resolution image, and the high resolution image. If the packet contains this information, then streaming algorithms can be developed to optimize the media delivery experience to users with diverse client devices.

Multiple Distortion Measures was published in ICASSP 2007 by Carri Chan, Susie Wee, and John Apostolopoulos.

The Future

The last part of the keynote focussed on experiences #2 and #3 to look at the impact of emerging applications on future packet networks.

Experience #2: Immersive, Conversational, and Worldwide

Delivering immersive, high-quality, worldwide experiences has a number of challenges for today's networks. The main problem is that network intelligence exists, but only in spots. For example:

QoS exists in spots, but is not guaranteed from beginning to end.
IPv6 exists in spots, but it is often tunneled over IPv4 and so is not available from beginning to end.
Significant congestion can occur in peering points between administrative domains, and it is very common for packets to traverse administrative domains many times in a single session.
Due to the sheer number of IP addressses, packets in countries such as India may go through many network address translations (NATs) before being delivered to the recipient.

Public & private domains

As a result, proprietary networks are being built to deliver guaranteed experiences. HP's Halo immersive collaboration experience is built on a proprietary network for that very reason.

In the long run, the right answer is to build out networks that contain IPv6 and QoS. However, until that occurs, there is likely to be a co-existence of public and proprietary networks.

This raises research opportunities of developing protocols and algorithms that improve media delivery over co-existing public and proprietary networks. This also motivates the need to develop packet labels that contain information that can be used by smarter network elements that understand them. And, this once again raises the design principle of designing the labels so that they are specific enough to be useful but general enough to be widely understood.

Experience #3: Pervasive, Personalized, Context-aware

Finally, I described Mediascapes as an example of pervasive, context-aware multimedia experiences. The main essence of Mediascapes is that it uses sensors to trigger multimedia experiences tied to your physical and personal context.

Sensing context in the network

This raises the question of using sensors to sense your context and getting the sensed context into packets that can be used by different applications and services. In the web world, the sensors may exist as GPS sensors, environmental sensors, or personal sensors. In the operator world the sensors may come through carrier-grade network elements as in IP Multimedia Subsystem (IMS) architectures. For example, IMS context can include location, presence, group lists, and subscriber info.

The key is to have the sensors provide context that is wrapped into packets in a manner that they can be easiliy used by applications and services. This raises the challenge of creating a semantic representation for sensed context. Again, like the packet labels, this must be designed in a manner that is both specific enough to be useful but general enough to be widely understood.

Acknowledgments

I'd like to take a moment to give special thanks to thank John Apostolopoulos, Carri Chan, Steve Froelich, Dave Penkler, Qibin Sun, and Zhishou Zhang for their contributions to various parts of this work!

Final note and questions

The audience was great and the talk seemed to generate lots of discussion throughout the workshop.

This was a fun topic to put together for the keynote and I'd like to develop it further. I'd love to hear your thoughts and ideas on any aspects of this.

What are your thoughts and comments on the life of a packet?
Did you attend the workshop and keynote? If so, what did you think?
I'd like to develop this further. Do you have any suggestions for improvements?

Please feel free to leave a URL with your comments.

Idea request: The Life of a Packet

susie.wee — Mon, 29 Oct 2007 07:53:00 GMT

I'll be giving a keynote talk at the Packet Video Workshop in Lausanne on Nov 12-13, 2007. The title of my talk is "The Life of a Packet". More specifically, I will be focusing on "The life of a media packet" and "the life of a packet as it relates to mobile & media experiences".

I have my ideas on what to talk about, but I'd love to hear your ideas as well.

What would you expect/want to be covered in this talk?
What would you expect/want to learn from this talk?

Thanks in advance for your inputs!

UPDATE!

It's been quiet... so I'll frame this up a little more. Here is the talk abstract. I'd love to hear your thoughts!

The Life of a Packet for Delivering Mobile & Media Experiences

People are mobile, and people interact through sight and sound. Multimedia communication can be used to provide mobile & media experiences that connect people with each other and with their environments. In these systems, media is delivered in packets. The use of media packets raises a number of questions that the research community and industry ha ve been tackling for decades.

How should media packets be transported across a packet network?
How can one deliver and adapt media packets for diverse media clients?
How does one protect media packets in a packet network?
What impact do globally distributed, immersive media environments have on media packet delivery systems?
What role does context play in next-generation mobile media experiences?

In this talk, we will consider these questions from the perspective of a user experience and from the perspective of a media packet.

One more update: Here is a link to the post that describes my keynote talk.

Hyperstimulated at HP

susie.wee — Thu, 06 Sep 2007 15:12:00 GMT

Sorry I've been away from blogging for a while. It's not because of a lack of ideas... It's the opposite. There are so many exciting things going on that I'm hyperstimulated- there's so much to write about!

Let me dive deeper into just a few of the many things that are going on in the company.

Personal Systems

HP's Personal Systems Group (PSG) run by Todd Bradley is really roaring ahead! PSG includes all our personal computing products, including consumer and business PCs, notebooks, handhelds, and televisions. They also have HP's new gaming business. I had the chance to attend Todd's quarterly staff meeting and see him in action. Todd is truly one of the most phenomenal leaders I have ever seen!

PSG is having their big product launch this week in NYC and they are announcing our new products. One very exciting product that catches my heart is Blackbird 002, the new gaming PC that bridges the gap between our HP and Compaq PCs and our VooDoo gaming PCs. Blackbird 002 was designed for the user experience with a focus on industrial design and performance. It has a sleek and sophisticated look and feel that is already resonating with hardcore gamers. It also has lots of technology inside. Famous bloggers Rahul Sood and Phil McKinney are two of the masterminds behind all of this.

HP Labs

I have a new boss at HP Labs, as Dick Lampman just retired on July 31 and Prith Banerjee started on August 1. At Dick's retirement party, George Dies (the HP Labs Director of Strategy) described Dick as a swan, who above water always appears graceful and calm but underwater is paddling powerfully to make things move ahead. This is the perfect description of Dick. He is an example of a great HP manager who showcases the HP Way. I'm glad I had the chance to work for him!

HP Labs is really kicking along now with its new leader, Prith Banerjee. Prith is a real change agent and he is starting us on an intensive effort to define the HP Labs strategic plan for the next 5 years. We have 4 teams of 30 people each coming from HP Labs and HP businesses working on creating the new strategic plan. I'm co-leading one of the four teams, and it's quite exciting to have the opportunity to define the New HP Labs!

Imaging and Printing

HP's Imaging and Printing Group (IPG) is also driving forward. They had their big product launch at the end of August and announced some very cool new products! They are also tactically moving forward on their Print 2.0 vision. IPG's CTO Patrick Scaglia has a great new Print 2.0 blog centered on this topic. A few years ago, Patrick was my boss at HP Labs; I know that he's a very interesting person with very interesting insights, so check out his blog! Also, HP blogger Eric Kintz has joined IPG to lead their marketing efforts. Check out his Digital Mindset blog where he discussed some of HP's coolest launched products!

IPG also announced new product offerings in its HP Halo business that offers a top-notch face-to-face conferencing experience. My team has been working closely with Halo team from day 1, while it was still just an investigation. Since then, Halo has become a real product and business unit with R&D, marketing, sales, operations, support, ... They are a great team, so it's fantastic to work with them as they continue to advance their product portfolio!

Phew- This is just a subset of all the things going on. It's been hard not to talk about them, so I'm glad they're announced! Can you see why I'm hyperstimulated?

Social web technology gone bad: Questioning anonymity

susie.wee — Wed, 15 Aug 2007 17:25:00 GMT

Alex Vorbau is a true believer of social technology. He uses social technology in all aspects of his daily life and his career, and he even focussed his work blog on Social Technology Innovation. But, he had a bad experience that made him question his beliefs. Take a look at his blog post on Thoughts on Anonymous Cowards. Alex started a legitimate web business with his wife. Someone out there decided that they didn't like his business, so much that they decided that they would try to sabotage it. They used the very same social web technologies that Alex develops, but they used it against him, and, they hid behind the anonymity of the internet to do it. This really is an example of social web technology gone bad.

I love how social web technology gives everyone a voice. I love that people can publish information with the click of a button. I love that people can post comments, both positive and negative, about what they read. I love the rich world-wide user-generated information pool that social web technology creates. And, I love that I can contribute to it with my own comments and my own blog posts. But what do we do when people abuse the freedom that social web technology has given them?

The social web world is an amazing self-governed community, self-governed by writers and readers. If I am honest in my posts, I feel I am rewarded by appreciation from blog readers. If I am dishonest in my posts, I feel I will get corrected by the blog community. And, if I admit to and correct my errors, I feel the community will forgive me and allow me to rebuild my credibility. I think blogging teaches you to be open, honest, and thoughtful. While as a blogger I am open about my identity, I actually like the fact that blog readers can post comments anonymously, especially when it helps them be honest about something they feel deeply about and can't express openly.

But what do you do when anonymity gets abused? What do you do when anonymous cowards attack well-meaning people?

Do we start developing social web technologies that don't allow anonymity? Do we start developing tools that limit or inhibit information sharing? Do we start developing tools to police the abusers?

I can't help think about the amount of social web innovation that is lost by attacking and demotivating well-meaning social web users and social web innovators. I wonder if the anonymous cowards realize the impact of what they are doing. What should we do?

ICME panel on HCI for Multimedia Communications

susie.wee — Mon, 09 Jul 2007 07:30:00 GMT

Last week I was on a panel on "HCI for Multimedia Communications" at the ICME 2007 conference in Beijing. The organizers and panelists were as follows:

Title

HCI for Multimedia Communications

Organizers

Qiong Liu, FX Palo Alto Laboratory, USA
Zhengyou Zhang, Microsoft Research, USA

Panelists

Zhengyou Zhang, Microsoft Research, USA
Tom Huang, University of Illinois at Urbana-Champaign, USA
John Weng, Michigan State University, USA
Susie Wee, Mobile & Media Systems Lab, HP Labs
Arnold Smeulders, University of Amsterdam, the Netherlands
John Smith, Watson Research Center, USA

We kicked off with each panelist stating their position on the topic, and then went into Q&A with the audience. The panelists contributed a variety of perspectives and the audience raised many interesting questions. Below are a few of my thoughts that I shared in the discussion.

Coupling experience with technology: When we think about multimedia communication, we should start from the user experience, and then think about the technology to enable it. There should be a close coupling between experience design and the enabling technologies.

Existing communication experiences: Technology can recreate existing communication experiences in situations where it is not possible. For example, technology can recreate effective face-to-face conversations for people who are geographically distant, as in HP's Halo virtual collaboration system.

New communication experiences: Technology can also be used to enable new comunication experiences for people. For example, instant messaging, blogging, photo sharing, and social networking are examples of new types of communication experiences that did not exist without the enabling technology.

Experience- Many simultaneous communication threads: HCI for multimedia communications should consider the new ways that people communicate, and specifically account for today's multitasking world where many communication threads are used simultaneously. For example, it is common for people to be reading email, browsing the web for research, getting instant messages, and getting phone calls simultaneously.

Experience- Share experiences instantly: As I was in Beijing, I constantly took photos and uploaded them to Facebook for my friends and family to see. They were able to share my Beijing experience while I was still there. We take this capability for granted today, but note that this was not so easy to do a few years ago.

Experience- Anyone can publish instantly: On top of sharing photos instantly, I can also publish instantly with my blog. In other words, I can create a publication at the click of button for the world to see without asking anyone for help or permission. This has a few implications:

I can publish on a wide variety of topics. At one time, I imagined all my publications would be technical. Now I have publications called "Blogging on the Beach", "Team sports and work", and "Top 10 Tips for How to Talk in Groups".
Since so many people like me can publish instantly, there is information overload and a lot of irrelevant information out there.
On the other hand, the fact that there are so many publishers greatly increases the chance that there is something out there that is useful to you and perhaps written by someone who shares your perspective or background.

People tag and vote: We are in a world of too much information and a scarcity of attention, so the question is how do you find what you want. Search and analysis are important. Tagging and voting can help. So, the advice for multimedia communications HCI is to use the proper balance of analysis and user feedback. Don't spend too much time analyzing if you have a situation where you can just go out and ask for user feedback. Since we are now in a world where people can tag and vote, use it when appropriate!

Measure quality of multimedia communications HCI by usage: Since there is a scarcity of attention, I think the best way to measure quality in multimedia communications HCI is by measuring usage. My advice is to build prototypes, attract users, and study their usage. You will learn a lot by doing this, including what is important, what are you missing, and what are users interested in.

How do we measure quality in multimedia communications HCI? Quality metrics should consider the multi-session nature of the communication experience that exists today.

How do we measure quality in multimedia communications HCI? Build prototypes. Attract users. Track usage (e.g., google analytics), user feedback (e.g., diggs), and quantitative metrics (analysis).

What will be an ideal interface for multimedia communication? An ideal interface will be simple, intuitive, and easy to use. You shouldn't need to read a manual to figure out how to use it.

Should the ideal interface have intelligence behind it? Only if it has predictable behavior. The interface must be predictable so users can learn to use it.

Should the interface have a human form? Not always. An HCI system for medical surgery should not have a false human interface.

What are your thoughts on the topic of HCI for multimedia communications?
Did you attend the panel discussion? If so, what did you think of it?
What advice do you have for the panel organizers and panelists?

Tags: HCI for multimedia communications, communication experience, human computer interaction, interface, IEEE ICME, Qiong Liu, Zhengyou Zhang, Tom Huang, John Weng, Susie Wee, Arnold Smeulders, John Smith, HP

Please feel free to leave a URL in your comments.

Are we communicating if we bypass our senses?

susie.wee — Sun, 01 Jul 2007 14:32:00 GMT

Communication is a person-to-person interaction. Communication involves one person conveying a message and another person interpreting the message. This can be done by talking/listening, gesturing/seeing, and writing/reading. Technology makes it possible for people to communicate even when they are far apart.

Sqchen raised some interesting points in his comment on my post on today's communication experience. He observed that communication uses the senses of sight and sound and asked what senses will be used next. He raised the possibility of having a communication chip implanted in you brain that allows you to automatically communicate with other people through their communication chips "without bothering our eyes", or more generally, without bothering our senses.

This makes me think about a few questions are:

Are we communicating if we bypass our senses?
Is communication about the information or about the experience?
What do you gain or lose if you communicate with communication chips instead of natural human senses?

Take a minute to think about your answers to the questions..........

OK, here's my take on the three questions- in reverse order:

What do you gain or lose if you communicate with communication chips instead of natural human senses?
If we communicated through communication chips instead of human senses:

We would be able to convey factual information more quickly and accurately.
We would be able to convey visual thoughts more easily and accurately. I'm specifically calling out visual information because pictures can be hard to describe in words, so this would be a great advantage.
We would not be able to convey or interpret feelings. Could a communication chip convey feelings and experiences? (I don't know how human senses relate to human feelings.)
We would lose the deep understanding that occurs when you think through and discuss a topic for a long time. Could a communication chip provide deep understanding?
We wouldn't have to work at formulating words around our thoughts. My thoughts could be zapped over to the other person, including the visual imagery that I have in my head. In fact, this would be very helpful for me right now since I'm in the design stage of a kitchen remodeling project-- this involves a lot of visualization and communication.
We would lose the learnings we get from going through the thought process of figuring out how to effectively communicate an idea. Thinking about how to communicate an idea, for example, when giving a presentation, leads to deeper thought and understanding. Could a communication chip instill deeper thought and understanding?
We would lose the person-to-person experiences that we get when spending time together. The person-to-person experience allows you to get to know each other, to get to know how well you get along with each other, how well you can brainstorm together, and how well you can work together.

Is communication about the information or about the experience?
I think conveying information is just one aspect of communication. I think a bigger part of communication lies in what you gain from the experience, such as gaining a deeper understanding from discussing different aspects of the topic and in getting to know the other person. I think human senses and intuition are a big part of communication.

Are we communicating if we bypass the senses?
In conclusion, at this point I'd say No. If we bypass our senses and use a communication chip to to communicate, we would be able to convey information, but we would not be able to communicate.

Well, that was my first take. I'd love to hear your thoughts on these questions!

Tags: communication experience, conveying information, deep thought and understanding, HP

Please feel free to leave a URL in your comments.

Today's communication experience: How far we've come

susie.wee — Wed, 27 Jun 2007 21:21:00 GMT

Alex wrote a very interesting post on sharing our lives in little pieces. He talks about how various new web technologies allow us to share pieces of our lives in different ways. Alex really captures the way I feel about social web tools such as blogging and Facebook- I feel that they give me a place to share a glimpse of my life that I might not have shared otherwise, and they let me do it easily. Blogging lets me share my thoughts through casual posts and Facebook lets me share my travels through casual photos.

This got me thinking about how far we've come with today's web technology and about how easy it is to communicate and share. It reminded me of my childhood memories of how my mom shared her new life in America with her parents who were back in Korea. My mom and dad were born and raised in Korea, and moved to the US after college in the 1960s. In those days, international travel was expensive and seldom done, international phone calls were expensive and seldom used, and pictures took days to get developed... and you had to wait until you finished the roll of film to get it developed! So, my mom communicated with her parents by writing letters and sending them through "air mail".

I remember my mom sitting in the spot in the living room where the sunlight shined through the windows, with a warm cup of coffee in her hands as she thought about what to write, and then handwriting a letter to her parents on special "air mail" paper, the kind with a red- and blue-striped border. I think she might have been restricted to writing everything on one page. Imagine the adventure of moving to a foreign country at a time when it was hard to communicate back home with your family and friends. Imagine wanting to share stories about your new life in America, your husband's career, your three kids' lives, and your adventures about living in a foreign country while learning a new culture and language. Imagine being restricted to a one-page, hand-written letter that takes one or two weeks to get delivered and waiting another one or two weeks to get a response.

Today, I probably travel to Asia more often than my mom wrote letters or called home. I can send an email or publish a blog post that can be read instantly by people around the world at the click of a button. I can snap a picture on my digital camera and share it easily and instantly. I can call my mom and dad from practically anywhere using my mobile phone. Let's take a moment to celebrate on how far we've come!

Now for the next phase. I am fortunate that my friends and loved ones live in places where they can get to a computer and get to the internet, though admittedly they do this with varying degrees of ease and regularity. But, I still know many people who have made the pilgrimmage to a new country but have friends and family in places where they can not connect to the web easily and regularly and in a cost-effective way.

Where are these people who can't get connected easily in a cost-effective way?
What can we do to help them get connected?
If we look ahead a few years, how will they be connected with the rest of the world?

Tags: communication experience, sharing, connected, web technology, air mail, blogging, Facebook, HP

Please feel free to leave a URL in your comments.

Blogging on an airplane

susie.wee — Tue, 22 May 2007 14:11:00 GMT

I'm on an airplane between Seoul and San Francisco as I make my way home back from Shanghai. I'm not on any airplane... I'm on Singapore Airlines. I'm not in any seat... I'm in the "new business class" which boasts the largest business class seating pod in the world. (On my last trip on this flight I was in coach and walked by these pods with envy... now I'm thrilled to be sitting in one.)

Back to the point. Singapore airlines has a mini-computer at each seat along with a Sun's Star Office suite which includes a word processor, so I thought I'd try blogging on an airplane using the setup. By the way, all the seats including coach have this.

So, what's the experience like... This is actually do-able if you're on a mission to use it (like I am). The keypad is not so great, since it is actually a gaming handset with a keyboard thrown in the middle, so I can't hold it on both sides and instead I have to grab it from above and below. Actually, as I use it more I realize that I can hold it vertically and type... hmmm... interesting!

The keypad doesn't seem to have backward and forward arrows to navigate through the text, so I can only move the cursor with the mouse. That's a bit annoying.

The screens in Singapore Air are quite which is very good for the experience.

You transfer the file through a USB stick as a word file. I took a look on my laptop and it worked! (Phew!)

Unfortunately they freeze it every time they make announcement, which is understandable, but annoying when they make the same announcement in three languages.

I am saving often, as I'm afraid that they will suddenly turn off the entertainment system without too much warning. Actually, they will provide plenty of warning but I will keep typing as long as I can.

Overall, I'd say it works!

I'll do it again. It's nice to get into the paradigm of not having to bring your computing device with you and using the computing that's provided by your environment.

Just touched down... gotta run!

Note: System is still working through taxiing! Now that I landed, it's time to switch to my iPAQ to check email...

Written on Friday, April 11, 2007 on a Singapore Airlines flight between Seoul and San Francisco. Left in its original, un-edited form.

Tags: blogging, airplane, Singapore Airlines, Sun StarOffice Suite, mobile blogging, HP

Please feel free to leave a URL in your comments.

Mscape makes wikipedia

susie.wee — Fri, 11 May 2007 01:02:00 GMT

"Mscape" just made made it into wikipedia! We'll have to beef up the entry with some more details, but I just wanted to share the news.

Here's a nice post at joystiq that gives a hands-on view of Mediascapes.

As you know, we're trying to build up the Mediascapes community through the Mscapers web site. I hope you get a chance to build your own Mediascape soon. If so, please do share your experiences.

We also just created an "Mscapers" group on Facebook and MySpace. Feel free to join in and contribute to the discussion there as well!

Tags: mediascape, mscape, mscaper, mediascapes, mscapers, mscapes, context-aware multimedia,wikipedia, facebook, myspace, HP

Please feel free to include a URL with your comments.

Mediascapes/Mscapes: A new medium and a new community web site

susie.wee — Wed, 09 May 2007 18:55:00 GMT

At the HP gaming summit last month [1,2] I hinted at the fact that Mediascapes had something in the works for May. I'm at the HP Mobility Summit in Shanghai and we just made the announcement. HP is creating a community web site for Mediascapes users and creators and we just opened up the beta release at www.mscapers.com!

Full disclosure: I manage the team and I love their work!

Mediascapes: A new medium

A Mediascape is a context-aware multimedia experience that allows you to trigger multimedia content based on your context such as location [3]. A key concept of Mediascapes is linking media to context, and we view this combination as a new medium. For example, there are images (pictures), audio (sound), video (time-sequenced pictures), and mediascapes (context-triggered multimedia). Mediascapes can be used to create games, educational tools, historical tours, and many other applications.

We'd like to get this new medium into widespread use and do it in as open a way as possible. Since we're trying to create a new medium, we figured that it deserves a community. So, we created the mscape web site as a place where people can go to create, share, and play Mediascapes and discuss their experiences.

Downloadable software

The Mscapers site has mscape software available for download, including:

Mscape toolkit (PC software): A Mediascape creation and management toolkit for use on your PC.
Mscape player (client software): A Mediascape player for use on your GPS-enabled mobile device running Windows Mobile 5.0 or newer.

Mscaper web site

The Mscapers site allows Mediascape creators to create and upload mediascapes and it allows Mediascape users to download and experience Mediascapes.

It also has Mscape wizards that let you create Mediascapes on the web site.

The Mscapers site also has discussion boards and a wiki that allows developers to discuss their experiences with the toolkit. It also has rating tools that allow users to rate and discuss the Mediascapes on the site.

Measures of success

The Mediascapes project is an HP Labs research project that has been underway for a few years. It is now a project in the HP Innovation Program Office [4], where we are looking for opportunities for productization.

Full disclosure: Our measure of success is the number of Mscapers. This includes both the number of people who download and play Mediascapes and the number of people who create and upload Mediascapes. So, we'd like to get as many Mscapers as possible!

Try it out!

Give it a try and let us know what you think! Please tell your friends about it, too!

What do you think of Mediascapes as a new medium?
Do you have suggestions on how we can establish Mediascapes as a new medium?
Do you have suggestions on how we can grow the number of Mscapers?

Tags: mediascape, mscaper, mscape, mediascapes, mscapers, mscapes, context-aware multimedia,a new medium, gaming, HP Labs, VooDoo, HP

Please feel free to include a URL with your comments.

JPSEC: Security for JPEG-2000 images

susie.wee — Sun, 29 Apr 2007 18:15:00 GMT

A technical thread that I've been working on for a number of years is media security for scalable media. As part of this I have been developing a media security standard for JPEG-2000 images called JPSEC. This is an international team effort as JPSEC was developed by technologists from around the world.

This week I've been attending the JPEG/MPEG international standardization meeting in San Jose, California. It's been a long process, and at this meeting we just received the notice of publication for JPSEC!!! This means that the standard is finalized and you will soon be able to get the JPSEC specification from the ISO the same way that you get the JPEG and MPEG coding specifications.

In this post, I provide an introduction to JPSEC with some background and motivation. I'll dive into more details in future posts.

JPSEC security tools

JPSEC is Part 8 of the JPEG 2000 family of standards. JPSEC specifies ways of applying security to JPEG-2000 coded images. The three types of security tools specified included in the normative part of the standard are: confidentiality, authentication, and integrity. While these are very standard security tools that are commonly used in many applications, the thing that is different in JPSEC is how these tools are applied to media. Specifically, JPSEC applies security tools to JPEG-2000 images in a media-aware way.

Media-aware security

The traditional way to apply security to media would be treat the media data like any other data file and secure the entire file in a media-unaware way. If a security tool such as encryption is applied in a media-unaware way, then any structure in the media data would be lost.

However, some structure in the media data can be quite useful. For example, scalable coding methods code images into a bitstream that has a structure that makes it easy to access to a low-resolution version of the image, without requiring one to decode or transcode the entire bitstream. If this image data is encrypted for confidentiality in a media-unaware way, then the ability to extract the low-resolution version would be lost, or it would require you to decrypt and then extract the low-resolution data. However, once the image is decrypted, it is no longer secure.

On the other hand, if media-aware security tools are used, then security can be applied in a way that preserves the useful structure in the media. JPSEC recognizes the fact that media data actually has some useful structure to it, so it specifies how to apply security tools to JPEG-2000 images in a media-aware way.

Scalable coding of media: The structure of JPEG-2000 image data

JPEG-2000 has a particularly useful structure because it was designed to be "scalable". While people often talk about the compression performance of JPEG-2000 vs. JPEG and other image coding methods. I think one of the biggest advantages of JPEG-2000 is its built-in scalability.

Scalable coding methods code media (images, video, or audio) in a manner that makes it easy to extract and decode different versions of it. For example, a scalably coded image can easily be decoded in high or low resolution. Decoding the image in high resolution involves decoding the entire bitstream. Decoding the image in low resolution simply involves extracting and decoding the low-resolution segments of the coded media data.

JPEG-2000 is a scalable image coding method. JPEG-2000 was designed in a way that makes it very easy to extract and decode a resolution, a tile, a color component, or a quality layer of the coded image. This can be done by simply scanning the bitstream, identifying and extracting the desired segments of the bitstream, and decoding those segments. This ability of transcoding to a lower resolution or quality level by simply grabbing portions of the image can be very useful for many applications.

Example application for scalable images

Let's say a server stores a very large, high-resolution image and a client with a smaller display would like to look at and virtually navigate around this image. Because the client's display resolution is much smaller than the original image resolution and because the bandwidth between the client and server may be limited, the options are to serve the client a small portion of the image in full resolution or the entire image in low resolution. In order to do this, the server would have to extract a portion of the image or extract a low resolution version of the image.

This can be achieved in different ways. If regular image coding is used, then the server would have to decode, process (select an area or downsample the image), and encode the image or transcode it accordingly. On the other hand, if scalable image coding is used, then it is very easy to extract portions of the image in different resolutions. Transcoding to lower resolutions or smaller tiles simply involves extracting the appropriate set of coded data. This requires very little computation, so it allows the server to support simultaneous image streaming sessions for many clients.

Adding security to media

A question that arises is what happens if end-to-end security is required for the application? For example, what if the application requires the image to be encrypted at the source and decrypted only by people who are allowed access? If this is required, then when the media data is transported between the sender and receiver, it must remain encrypted at all times, including when it is stored on the server.

When the media data is encrypted, what happens to the nice property of being able to adaptively stream portions of the high-resolution original image to lower-resolution clients? If the media is encrypted in a media-unaware way, then this property is lost, or the only way to do adaptively stream is by decrypting the image, but this breaks the end-to-end security of the system.

On the other hand, if media-aware security is used, then the security tools can be applied to the media data in a media-aware way in order to preserve the structure of the protected media and allow the server to adaptively stream portions of the protected media data.

JPSEC media-aware security tools

JPSEC was designed to provide media-aware security tools for JPEG-2000 images. It recognizes the structure of the JPEG-2000 image data, and it secures the media data within that structure. Specifically, it recognizes where the media data is located and which parts of the data correspond to which image components (tile, resolution, quality layer, color component, or image subband). It then allows security tools to be applied to subsets of the image data, and specifies the signaling data that must be included in the protected bitstream to allow the protected subsets of data to be extracted. In other words, JPSEC simultaneously allows mid-network transcoding and end-to-end security!

I hope this provides an introduction to JPSEC along with some background and motivation. I'll dive into more details in future posts! I'll be evolving this into a publication, so please let me know if you have any comments on the description or the text.

What do you think about JPSEC?

Any questions or comments?

Tags: JPSEC, JPEG 2000, scalable coding, security, image security, media security, media-aware security, media-unaware security, end-to-end security, transcoding, media adaptation, secure transcoding, HP

Please feel free to include a URL in your comments.