\chapter{Semantic Clipboard}
\label{clipboard}
\section{Inspiration}
This draws inspiration from the work done on `XHTML Documents with Inline, Policy-Aware Provenance' \cite{harvey-thesis} by Harvey Jones. Jones developed a document format that can represent information about the sources of its content, a method of excerpting from these documents that allow programs to trace the excerpt back to the source, a Creative Commons reasoning engine which calculates the appropriate license for the composite document, and a bookmarklet that uses all these components to recommend permissible licenses. However, there are several issues with this tool:
\begin{enumerate}
\item All the source documents that the user needs to copy from, have to be annotated with a special document fragment ontology.
\item It can only be used for text copying and pasting.
\item The implementation of the \emph{Paste Operation} is limited to inserting copied XHTML in the top level of the document, and does not allow copying inside existing document fragments.
\end{enumerate}
The Semantic Clipboard described in this chapter tries to address these issues. Specifically:
\begin{enumerate}
\item The reliance of an external document fragment ontology is completely eliminated.
\item Image reuse is supported. But this tool this currently does not support text copying.
\item Since the Semantic Clipboard utilizes the operating system's clipboard, the image with the associated license metadata in XHTML can be copied to any editor application at any location as source text.
\end{enumerate}
The only requirement for the Semantic Clipboard to work is that the license information about the work to be expressed in RDFa in the source documents.
\section{Enabling Awareness of Licenses}
\label{lic-aware}
License properties describe the permissions, prohibitions, and requirements of a license. Permissions declare a permission given by the license, above and beyond what default copyright law allows. Prohibitions prohibit a particular use of the work, specifically affecting the scope of the permissions provided by a permission. Requirements describe actions required by the user when making use of the permissions \cite{license-properites}.
CC offers a handful of licenses that are applicable to most use and restriction scenarios. However, if these licenses are buried in the HTML or if the end user does not understand the CC license icons that signal the accepted use of the content, the user may not be aware of the licensing options available to her.
Also it would be useful to know which of the images that the user sees on a given Web page can be used for commercial purposes, which images allow modifications, which images have to be reused under the same license, etc. Figure \ref{fig-license-venn} shows all the different CC license types grouped in to the most common \emph{Use} and \emph{Restriction} categories. Based on these categories, a menu is implemented to enable the user to easily select the images based on the user's intentions.
The \emph{Use} categories available in the Semantic Clipboard menu are as follows:
\begin{enumerate}
\item Any Use: This category includes images licensed under `CC0' or images that are under the `Public Domain Certification'. These are images where no rights are reserved, and reuse is allowed for any purpose without restriction under copyright.
\item Commercial Use: This category includes images that essentially have no `Non-Commercial' use clause in the license. We imagine this option to be useful to individuals or organizations who would like to reuse existing images on the Web without having to worry about any legal ramifications.
\item Allow Modifications: This category includes images which do not have the `No-Derivatives' clause in the CC license. These images can be reused in other free culture media.
\end{enumerate}
\begin{figure}[!h]
\centerline{\epsfig{file=images/license_venn_diagram_new.jpg, height=4in}}
\caption{Venn Diagram for the Acceptable Use and Restricted Use of Content Depending on the CC License}
\label{fig-license-venn}
\end{figure}
The Venn diagram given in Figure \ref{fig-license-venn} also includes few \emph{Restriction} categories represented by the areas colored in gray. These are also incorporated in the Semantic Clipboard menu to give users more awareness about which images cannot be used for a specific purpose based on their intention:
\begin{enumerate}
\item No Use: These are images that have explicitly given a copyright notice indicating that all rights are reserved.
\item Use Same License: This category includes images which are under a CC `Share-Alike' license. Users should take care when combining such images with some other licensed works.
\item No Modifications and Commercial Use: These restrictions specify that no derivatives are allowed and the image can only be copied for commercial use. However, this allows the user to redistribute the composite work under a different license.
\end{enumerate}
In the case of when there is no license, we make no decision. The original content creator may have had the intention to share the work with others. But since there is no indication of it expressed explicitly, copyright law specifies that those images are copyrighted and are not to be used. But we do not make this assertion, as the original content creator's intention might have been otherwise.
To make these allowed uses and restrictions clearer, the Semantic Clipboard gives visual cues to the user to signal what can be copied based on the intention of the user as described in Section \ref{ui-enhancer}.
\section{Design and Implementation}
This is a Firefox Web browser based tool integrated with the Tabulator, a linked data browser that can be installed as a Firefox extension ~\cite{tab}. The primary goal of this tool is to let users reuse content with minimal effort.
\begin{figure}[!h]
\centerline{\epsfig{file=images/semclip_design_new.jpg, width=1\linewidth}}
\caption{Design of the Semantic Clipboard}
\label{fig-semclip-design}
\end{figure}
When a Web page loads, the Semantic Clipboard registers the parts of the DOM that include the content intended to be reused along with their provenance and license metadata expressed in RDFa. Then it creates an XHTML snippet according to the Creative Commons specification for proper attribution. The design overview of the Semantic Clipboard is given in Figure \ref{fig-semclip-design}. The following sections describe what each of the components are designed to do.
\subsection{RDFa Extractor}
\label{rdfa-extract}
\begin{figure}[!h]
\centerline{\epsfig{file=images/image_with_rdfa.jpg, height=4in}}
\caption{Sample Image with Metadata about it Expressed in RDFa}
\label{fig-sample-image-with-rdfa}
\end{figure}
This module will extract all the RDFa embedded in the HTML page the user browses. For example, consider the image shown in Figure \ref{fig-sample-image-with-rdfa}. The creator of the image has embedded the image, and given some information regarding it in both human readable and machine readable formats. If you look more carefully at the source of this embedded image in the document, you would be able to see the XHTML snippet adorned with RDFa as shown in Figure \ref{fig-sample-metadata-expression}. This describes the image in terms of the creator, some tags, the date the photo was created, latitude, longitude, and the license it is under. All the \emph{license metadata} about the image are given in red and the \emph{other metadata} are given in blue.
The RDFa Extractor implemented in the Semantic Clipboard uses the JQuery Javascript Library \cite{jquery} to parse the embedded RDFa in Web pages. JQuery simplifies HTML DOM traversing, event handling, and Ajax interactions for rapid web development. By using a JQuery specific `selector' for the attributes, the RDFa from web pages are extracted. For example, the extracted RDF triples in the above XHTML snippet in Notation 3 serialization \cite{n3logic} is shown in Figure \ref{fig-sample-metadata-n3}.
For this particular application we are only concerned about the license information associated with the images. Therefore, out of all the extracted metadata, only the license information will be kept in an in-memory `RDFa License Store' to be used in generating the attribution XHTML snippet.
\subsection{UI Enhancer}
\label{ui-enhancer}
We have implemented several menu options in the Firefox browser to select licensed images with the proper intention. These are based on the allowed \emph{Use} and \emph{Restriction} categories as discussed in Section \ref{lic-aware}. The options available within the tool are as given in Figure \ref{fig-menu-selection}.
\begin{figure}[!h]
\centerline{\epsfig{file=images/menu_selection_new.jpg, height=3in}}
\caption{Firefox Menu to Specify User Purpose to See What is Available}
\label{fig-menu-selection}
\end{figure}
Once the user selects the corresponding menu option, if there are any images that satisfy the criteria, those images will be overlaid with a small `text notice'. For example, in Figure \ref{fig-menu-selection}, it shows that the user wants to see all the images on the Web page she's browsing which can be used for commercial purposes. Once this option is selected, all the images on the page which have CC-Zero, CC-BY, CC-BY-SA, CC-BY-ND, or CC-BY-ND-SA licenses (essentially all the licenses which does not have a NC clause) will be overlaid with the `text notice'. An example is shown in Figure \ref{fig-output-selection}.
\begin{figure}[!h]
\centerline{\epsfig{file=images/output_selection.jpg, width=4in}}
\caption{Image Overlaid With the Purpose Information}
\label{fig-output-selection}
\end{figure}
In addition, more subtle visual cues such as tool tip texts will be displayed if the image has a CC license that allows it to be reused. A example of such a tool tip text is shown in Figure \ref{fig-tool-tip}. Note that this tooltip will not be displayed if the the image already has a `title' \footnote[1]{The tooltip displays whatever the string that is given to the `title' property of the image.}. We hope that these visual cues will enable the end-user to select the images for copying in a more policy aware manner.
\begin{figure}[!h]
\centerline{\epsfig{file=images/tool_tip_new.jpg, height=0.75in}}
\caption{Tooltip on an Image giving Visual Cues as to whether the Image can be Copied or Not}
\label{fig-tool-tip}
\end{figure}
\begin{figure}[!h]
\centerline{\epsfig{file=images/image_right_click.jpg, width=4in}}
\caption{Right Click Context Menu on an Image}
\label{fig-how-to-copy}
\end{figure}
\subsection{Attribution XHTML Constructor}
\label{xhtml-constructor}
The user can issue a copy instruction on a particular image by right-clicking on the image and selecting the context menu option `Copy Image with License' as shown in Figure \ref{fig-how-to-copy}. Then, based on the image URI that acts as an index to all the subjects gleaned from the page, the RDFa license triples corresponding to this subject (i.e. the URI of the image that is being copied) are retrieved from the RDFa License Store. This constitutes the triples with the predicates \emph{attributionName} and the \emph{attributionURL}. Based on these triples, the attribution XHTML snippet is constructed. An example attribution XHTML snippet with the image embedded is given in Figure \ref{fig-attribution-xhtml}. This XHTML snippet will be copied to the system clipboard using a Mozilla specific component (namely `@mozilla.org/widget/clipboardhelper;1' that implements nsIClipboardHelper) \cite{mozclip}. This allows the user to paste it in to any application that accepts ASCII text \emph{data flavor} as input.
%Reasons whether the content can be used based on the source and the destination license terms. Prepares the content and the license metadata in a suitable manner in to be pasted right in to the target DOM.
\subsection{User Interface}
The Firefox browser XUL \cite{xul} overlay is over-written such that the context menu that appears when images are right clicked will have the extra menu option `Copy Image with License' as shown in Figure \ref{fig-how-to-copy}. Once this option is selected, the license data in the RDFa associated with the image will be extracted as described in Section \ref{rdfa-extract}, the attribution XHTML will be constructed as described in Section \ref{xhtml-constructor}, and this XHTML snippet will be made available in the system clipboard. Also, as mentioned in Section \ref{ui-enhancer}, the user interface in the browser can be overlaid with the necessary information about the allowed uses and restrictions of the images based on the licenses that are attached with those. This will assist the user in making an informed decision about what image(s) the user can copy for a specific purpose.
\begin{figure}[!h]
\begin{alltt}
\singlespace{
}
\end{alltt}
\caption{Metadata Description of Image given in Figure \ref{fig-sample-image-with-rdfa}}
\label{fig-sample-metadata-expression}
\end{figure}
\begin{figure}[h]
\begin{alltt}
\singlespace{
\textcolor{red}{ cc:attributionName "Oshani"} ;
\textcolor{red}{cc:attributionURL };
dc:creator ;
dc:date "2009-04-26"^^xsd:date ;
dc:description "Plaza Mayor, Madrid." ;
dc:subject "King Philips III", "Madrid", "Plaza Mayor", "Spain", "Statue" ;
\textcolor{red}{dc:license } ;
geo:latitude "40.4156" ;
geo:longitude "-3.7074" .
}
\end{alltt}
\caption{RDF Metadata about the Image in Notation 3 Serialization }
\label{fig-sample-metadata-n3}
\end{figure}
\begin{figure}[h]
\begin{alltt}
\singlespace{
}
\end{alltt}
\caption{Constructed Attribution XHTML Snippet}
\label{fig-attribution-xhtml}
\end{figure}
\textcolor{blue}{Plaza Mayor, Madrid.} 
By