<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.15b.dev: http://docutils.sourceforge.net/" />
<meta name="version" content="S5 1.1" />
<title>Wounded hero revived</title>
<style type="text/css">
/*
:Author: David Goodger (goodger@python.org)
:Id: $Id$
:Copyright: This stylesheet has been placed in the public domain.
Default cascading style sheet for the HTML output of Docutils.
See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/
/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
border: 0 }
table.borderless td, table.borderless th {
/* Override padding for "table.docutils td" with "! important".
The right padding separates the table cells. */
padding: 0 0.5em 0 0 ! important }
.first {
/* Override more specific margin styles with "! important". */
margin-top: 0 ! important }
.last, .with-subtitle {
margin-bottom: 0 ! important }
.hidden {
display: none }
.subscript {
vertical-align: sub;
font-size: smaller }
.superscript {
vertical-align: super;
font-size: smaller }
a.toc-backref {
text-decoration: none ;
color: black }
blockquote.epigraph {
margin: 2em 5em ; }
dl.docutils dd {
margin-bottom: 0.5em }
object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] {
overflow: hidden;
}
/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
font-weight: bold }
*/
div.abstract {
margin: 2em 5em }
div.abstract p.topic-title {
font-weight: bold ;
text-align: center }
div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
margin: 2em ;
border: medium outset ;
padding: 1em }
div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
font-weight: bold ;
font-family: sans-serif }
div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title, .code .error {
color: red ;
font-weight: bold ;
font-family: sans-serif }
/* Uncomment (and remove this text!) to get reduced vertical space in
compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
margin-bottom: 0.5em }
div.compound .compound-last, div.compound .compound-middle {
margin-top: 0.5em }
*/
div.dedication {
margin: 2em 5em ;
text-align: center ;
font-style: italic }
div.dedication p.topic-title {
font-weight: bold ;
font-style: normal }
div.figure {
margin-left: 2em ;
margin-right: 2em }
div.footer, div.header {
clear: both;
font-size: smaller }
div.line-block {
display: block ;
margin-top: 1em ;
margin-bottom: 1em }
div.line-block div.line-block {
margin-top: 0 ;
margin-bottom: 0 ;
margin-left: 1.5em }
div.sidebar {
margin: 0 0 0.5em 1em ;
border: medium outset ;
padding: 1em ;
background-color: #ffffee ;
width: 40% ;
float: right ;
clear: right }
div.sidebar p.rubric {
font-family: sans-serif ;
font-size: medium }
div.system-messages {
margin: 5em }
div.system-messages h1 {
color: red }
div.system-message {
border: medium outset ;
padding: 1em }
div.system-message p.system-message-title {
color: red ;
font-weight: bold }
div.topic {
margin: 2em }
h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
margin-top: 0.4em }
h1.title {
text-align: center }
h2.subtitle {
text-align: center }
hr.docutils {
width: 75% }
img.align-left, .figure.align-left, object.align-left, table.align-left {
clear: left ;
float: left ;
margin-right: 1em }
img.align-right, .figure.align-right, object.align-right, table.align-right {
clear: right ;
float: right ;
margin-left: 1em }
img.align-center, .figure.align-center, object.align-center {
display: block;
margin-left: auto;
margin-right: auto;
}
table.align-center {
margin-left: auto;
margin-right: auto;
}
.align-left {
text-align: left }
.align-center {
clear: both ;
text-align: center }
.align-right {
text-align: right }
/* reset inner alignment in figures */
div.align-right {
text-align: inherit }
/* div.align-center * { */
/* text-align: left } */
.align-top {
vertical-align: top }
.align-middle {
vertical-align: middle }
.align-bottom {
vertical-align: bottom }
ol.simple, ul.simple {
margin-bottom: 1em }
ol.arabic {
list-style: decimal }
ol.loweralpha {
list-style: lower-alpha }
ol.upperalpha {
list-style: upper-alpha }
ol.lowerroman {
list-style: lower-roman }
ol.upperroman {
list-style: upper-roman }
p.attribution {
text-align: right ;
margin-left: 50% }
p.caption {
font-style: italic }
p.credits {
font-style: italic ;
font-size: smaller }
p.label {
white-space: nowrap }
p.rubric {
font-weight: bold ;
font-size: larger ;
color: maroon ;
text-align: center }
p.sidebar-title {
font-family: sans-serif ;
font-weight: bold ;
font-size: larger }
p.sidebar-subtitle {
font-family: sans-serif ;
font-weight: bold }
p.topic-title {
font-weight: bold }
pre.address {
margin-bottom: 0 ;
margin-top: 0 ;
font: inherit }
pre.literal-block, pre.doctest-block, pre.math, pre.code {
margin-left: 2em ;
margin-right: 2em }
pre.code .ln { color: grey; } /* line numbers */
pre.code, code { background-color: #eeeeee }
pre.code .comment, code .comment { color: #5C6576 }
pre.code .keyword, code .keyword { color: #3B0D06; font-weight: bold }
pre.code .literal.string, code .literal.string { color: #0C5404 }
pre.code .name.builtin, code .name.builtin { color: #352B84 }
pre.code .deleted, code .deleted { background-color: #DEB0A1}
pre.code .inserted, code .inserted { background-color: #A3D289}
span.classifier {
font-family: sans-serif ;
font-style: oblique }
span.classifier-delimiter {
font-family: sans-serif ;
font-weight: bold }
span.interpreted {
font-family: sans-serif }
span.option {
white-space: nowrap }
span.pre {
white-space: pre }
span.problematic {
color: red }
span.section-subtitle {
/* font-size relative to parent (h1..h6 element) */
font-size: 80% }
table.citation {
border-left: solid 1px gray;
margin-left: 1px }
table.docinfo {
margin: 2em 4em }
table.docutils {
margin-top: 0.5em ;
margin-bottom: 0.5em }
table.footnote {
border-left: solid 1px black;
margin-left: 1px }
table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
padding-left: 0.5em ;
padding-right: 0.5em ;
vertical-align: top }
table.docutils th.field-name, table.docinfo th.docinfo-name {
font-weight: bold ;
text-align: left ;
white-space: nowrap ;
padding-left: 0 }
/* "booktabs" style (no vertical lines) */
table.docutils.booktabs {
border: 0px;
border-top: 2px solid;
border-bottom: 2px solid;
border-collapse: collapse;
}
table.docutils.booktabs * {
border: 0px;
}
table.docutils.booktabs th {
border-bottom: thin solid;
text-align: left;
}
h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
font-size: 100% }
ul.auto-toc {
list-style-type: none }
</style>
<!-- configuration parameters -->
<meta name="defaultView" content="slideshow" />
<meta name="controlVis" content="hidden" />
<!-- style sheet links -->
<script src="ui/small-black/slides.js" type="text/javascript"></script>
<link rel="stylesheet" href="ui/small-black/slides.css"
type="text/css" media="projection" id="slideProj" />
<link rel="stylesheet" href="ui/small-black/outline.css"
type="text/css" media="screen" id="outlineStyle" />
<link rel="stylesheet" href="ui/small-black/print.css"
type="text/css" media="print" id="slidePrint" />
<link rel="stylesheet" href="ui/small-black/opera.css"
type="text/css" media="projection" id="operaFix" />
</head>
<body>
<div class="layout">
<div id="controls"></div>
<div id="currentSlide"></div>
<div id="header">
</div>
<div id="footer">
<h1>Wounded hero revived</h1>
</div>
</div>
<div class="presentation">
<div class="slide" id="slide0">
<h1 class="title">Wounded hero revived</h1>
<p>Lessons learned from porting M2Crypto to Py3k</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field"><th class="field-name">Author:</th><td class="field-body">Matěj Cepl <<a class="reference external" href="mailto:mcepl%40cepl.eu">mcepl<span>@</span>cepl<span>.</span>eu</a>></td>
</tr>
</tbody>
</table>
</div>
<div class="slide" id="origins">
<h1>Origins</h1>
<ul class="simple">
<li>Mitch Kapor sold Lotus to IBM and decided to perpetrate good.
One of his projects was <a class="reference external" href="https://en.wikipedia.org/wiki/Chandler_(software)">Chandler</a>.</li>
<li>Project is <a class="reference external" href="https://gitlab.com/mcepl/chandler">gone</a> The only remainder of the project is
<a class="reference external" href="https://gitlab.com/m2crypto/m2crypto/">M2Crypto</a>, full Python binding for OpenSSL.</li>
</ul>
<p class="handout">Once upon a time, one Mitch Kapor, who sold Lotus to IBM, and
with money he got decided to perpetrate good. He did many
truly good things for the computer world, was co-founder of
EFF, helped with Mozilla Foundation, but he also founded
rather unsuccessful project, a Python-based universal PIM,
called <a class="reference external" href="https://en.wikipedia.org/wiki/Chandler_(software)">Chandler</a>.</p>
<p class="handout">Subversion of the project has been mirrored couple of times,
one by me.</p>
</div>
<div class="slide" id="id1">
<h1>M2Crypto</h1>
<ul class="simple">
<li>M2Crypto was maintained by <a class="reference external" href="https://www.heikkitoivonen.net/">Heikki Toivonen</a> few years after
Chandler folded, but his last release 0.21.1 was from 2011.</li>
<li>Maintenaned in Red Hat by <a class="reference external" href="https://github.com/mtrmac">Miloslav Trmač</a>.</li>
<li>I took over the project in May 2015.</li>
</ul>
<p class="handout">Maintenaned in Red Hat by <a class="reference external" href="https://github.com/mtrmac">Miloslav Trmač</a>, who collected
all patches in RHEL package.</p>
<p class="handout">I took over the project in May 2015 with the intention just
to publish all patches and be a point of contact for any
issue reports. I haven’t expected much activity, because
package was very silent in RHEL.</p>
</div>
<div class="slide" id="strengths-and-weaknesses">
<h1>Strengths and weaknesses</h1>
<!-- Strengths -->
<ul class="simple">
<li>Backed up by stable C library</li>
<li>Rather large coverage of OpenSSL API</li>
<li>Surprisingly widespread use</li>
<li>Large test suite</li>
</ul>
<!-- Weakness -->
<ul class="simple">
<li>Unknown issues</li>
<li>Python 3</li>
<li>M2Crypto API copies OpenSSL too closely</li>
<li>Support for Mac OS X and Windows (not mentioning <tt class="docutils literal">*BSD</tt>) was
broken.</li>
</ul>
<p class="handout">Backed by … comparing to PyCrypto and other horrors.</p>
</div>
<div class="slide" id="opportunities-threats">
<h1>Opportunities & Threats</h1>
<!-- Opportunities -->
<ul class="simple">
<li>Satisfying current user base</li>
<li>Replacing horrors like PyCrypto</li>
<li>Goal of maitenance is to maintain API</li>
<li>Extend support on non-Linux platforms</li>
</ul>
<!-- Threats -->
<ul class="simple">
<li>Python <tt class="docutils literal">ssl</tt> module</li>
<li>Python <a class="reference external" href="https://github.com/pyca/cryptography">cryptography</a></li>
</ul>
<p class="handout">Distribution bug tracker (especially an enterprise one) is
not a good measure of the real state of use and quality of
package.</p>
<p class="handout">There are apparently many programmers for custom software,
who use M2Crypto (still it is one of the most complete
bindings for OpenSSL).</p>
<p class="handout">Threats as a “competing” projects, which may replace
M2Crypto.</p>
</div>
<div class="slide" id="unicode">
<h1>Unicode</h1>
<ul class="simple">
<li>The biggest problem of all Python 2 programs: complete
confusion between py2k <tt class="docutils literal">str</tt> means py3k <tt class="docutils literal">str</tt> and when
<tt class="docutils literal">bytes</tt>.</li>
<li>There are numerous uses of both in M2Crypto, because of course
both strings and binary data are present in all functions of
OpenSSL.</li>
</ul>
</div>
<div class="slide" id="strategy">
<h1>Strategy</h1>
<ul class="simple">
<li>Type Hints</li>
<li>CI</li>
<li>Extension of platform support</li>
</ul>
</div>
<div class="slide" id="type-hints">
<h1>Type Hints</h1>
<ul>
<li><p class="first"><a class="reference external" href="https://www.python.org/dev/peps/pep-0484/">PEP 484</a> providing <strong>optional</strong> type annotations. Quite
controversial, but clearly very useful for libraries</p>
</li>
<li><p class="first">Native for Python >= 3.5, but supports py2k compatible syntax:</p>
<pre class="literal-block">
def sum(x, y):
# type: (int, int) -> int
return x + y
</pre>
</li>
<li><p class="first">Especially useful for our situation: marking types helps us to
analyze what invidivual py2k <tt class="docutils literal">str</tt> actually mean.</p>
</li>
</ul>
</div>
<div class="slide" id="c-api">
<h1>C API</h1>
<ul class="simple">
<li>All Unicode/bytes translation happens on C level as well</li>
<li>Based on <tt class="docutils literal">swig</tt>, which fortunately natively supports
<tt class="docutils literal"><span class="pre">--py3</span></tt>.</li>
<li>Also need to support two versions of OpenSSL API, 1.1 and
older.</li>
<li>Minimize use of <tt class="docutils literal">#ifdef</tt> s and rather use included shims
for missing functions.</li>
</ul>
</div>
<div class="slide" id="c-shims-of-missing-functions">
<h1>C shims of missing functions</h1>
<ul class="simple">
<li>For OpenSSL < 1.1</li>
<li>For Python 2<ul>
<li><tt class="docutils literal">PyLong_FromLong()</tt> and <tt class="docutils literal">PyUnicode_AsUTF8()</tt> just simple
<tt class="docutils literal">#define</tt> s.</li>
<li>All Pythons >= 2.6 contain whole set of Py3k function stubs
in <tt class="docutils literal">bytesobject.h</tt>.</li>
</ul>
</li>
</ul>
</div>
</div>
</body>
</html>
