Software Development¶
https://en.wikipedia.org/wiki/Software_development
https://en.wikipedia.org/wiki/Software_development_process
https://en.wikipedia.org/wiki/Category:Software_design_patterns
- https://en.wikipedia.org/wiki/Architectural_pattern_(computer_science)
- http://martinfowler.com/eaaCatalog/
https://en.wikipedia.org/wiki/Software_engineering
Computer Science¶
https://en.wikipedia.org/wiki/Computer_science#Applied_computer_science
https://en.wikipedia.org/wiki/Outline_of_computer_science
https://en.wikipedia.org/wiki/Category:Computer_science
Topics:
- https://en.wikipedia.org/wiki/Abstraction_(computer_science)
- https://en.wikipedia.org/wiki/Category:Algorithms
- Computer Engineering
- Data Structures
- Operating Systems
- Software Development (applied CS)
- Data Science (applied CS)
- Knowledge Engineering (applied CS)
- Science (applied CS with/for/on behalf of a domain)
Computer Science Curricula¶
- Curricula
- Criteria
- https://en.wikipedia.org/wiki/Criterion
- https://en.wikipedia.org/wiki/Criterion_(disambiguation)#Science_and_mathematics
- https://en.wikipedia.org/wiki/Information_criterion
- “Common Criteria” – Information Security platform specification
- “Common Core” – US education standard: https://en.wikipedia.org/wiki/Common_Core_State_Standards_Initiative
- ROI, ROI
- Rubric
- https://en.wikipedia.org/wiki/Rubric_(academic)
- A rubric can be expressed with a matrix data structure.
- A rubric can be expressed as an RDF graph in e.g. RDFa or JSON-LD.
Computer Science criteria, guidelines, and curricula:
ACM Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science (December 20, 2013)
https://www.acm.org/education/CS2013-final-report.pdf
- AL – Algorithms and Complexity (p.55)
- AR – Architecture and Organization (p.62)
- CN – Computational Science (p.68)
- DS – Discrete Structures (p.76)
- GV – Graphics and Visualization (p.82)
- HCI – Human-Computer Interaction (p.89)
- IAS – Information Assurance and Security (p.97)
- IM – Information Management (p.112)
- IS – Intelligent Systems (p.121)
- NC – Networking and Communication (p.130)
- OS – Operating Systems (p.135)
- PBD – Platform-Based Development (p.142)
- PL – Programming Languages (p.155)
- SDF – Software Development Fundamentals (p.167)
- SE – Software Engineering (p.172)
- SF – Systems Fundamentals (p.186)
- SP – Social Issues and Professional Practice (p.192)
ACM Computer Science Curriculum 2008: An Interim Revision of CS 2001
http://www.acm.org/education/curricula/ComputerScience2008.pdf
Programmer Competency Matrix
http://sijinjoseph.com/programmer-competency-matrix/
https://competency-checklist.appspot.com/
https://github.com/hltbra/programmer-competency-checklist
- computer science
- data structures
- algorithms
- systems programming
- software engineering
- source code and version control
- build automation
- automated testing
- programming
- problem decomposition
- systems decomposition
- communication
- code organization within a file
- source tree organization
- code readability
- defensive coding
- error handling
- IDE
- API
- frameworks
- requirements
- scripting
- database
- knowledge
- tool knowledge
- languages exposed to
- codebase knowledge
- knowledge of upcoming technologies
- platform internals
- books
- blogs
- experience
- languages with professional experience
- platforms with professional experience
- years of professional experience
- domain knowledge
- computer science
Computer Science Courses¶
Computer Science Exercises¶
Exercises, Problems
Project Management¶
- https://en.wikipedia.org/wiki/DMAIC
- https://en.wikipedia.org/wiki/Kaizen
- https://en.wikipedia.org/wiki/Systems_development_life_cycle
- https://en.wikipedia.org/wiki/Software_project_management
- https://en.wikipedia.org/wiki/Software_development_process
Waterfall¶
Agile¶
- https://en.wikipedia.org/wiki/Lean_software_development
- http://epf.eclipse.org/wikis/epfpractices/
- http://agilemodeling.com/essays/bestPractices.htm
Stand Up Meeting¶
Note
Meeting participants need not actually stand; as in a Digital Stand Up Meeting.
Three Questions¶
- What did I accomplish yesterday?
- What will I do today?
- What obstacles are impeding my progress?
Digital Stand Up Meeting¶
A Digital Stand Up Meeting is a meeting conducted through digital means of communication; rather than through traditional in-person communication.
Pros:
- Links: Realtime collaborative documents (live docs), team chat logs, and issue/ticket comment threads have URIs, are easier to remember, and more to the point.
- Easier to prepare for
- More time to read, think, and respond
- More comfortable
- Easier to get up to date with
Cons:
- It’s not everyone in the same room
- “Shouldn’t this all be in the issue tracker?”
- Mailing Lists, Forums, SIOC <- Links to source, docs, issues
- Bugs, User Stories -> Issue Tracker
- Daily Stand Up Meeting log / minutes ->
Digital Stand Up Meeting Process Checklist:
[ ] Create a collaboratively edited document (“live doc”, “doc”)
Realtime collaborative editors: Etherpad Lite, Google Docs
An example of Three Questions for stand up meetings (in Markdown, with URIs, for GitHub, BitBucket):
# team-name/project-name Daily Stand Up Log ## 2015-01-01 ### @username1 1. yesterday: #1, #2, http:// 2. today: #2 3. obstacles: - ### @username2 1. yesterday: #3, #2, https:// 2. today: #2 3. obstacles: -
[ ] Share the link to the live doc with the team
- [ ] Email, Text Chat
- [ ] Meeting/event invitations (iCal)
- [ ] Chat (Text, Voice, Video)
- Text
- Slack
- Mattermost
- HipChat
- Gitter
- XMPP
- IRC
- Voice/Video (cross-platform): Google Hangouts, Skype, Cisco WebEx,
WebRTC
- Live streaming
- YouTube
- Live streaming
- Text
Extreme Programming¶
Unified Process¶
- “Rational Unified Process” (RUP)
- OpenUP is an Open Source Unified Process
Scrum¶
The Daily Scrum (EPF Wiki Entry)
http://epf.eclipse.org/wikis/scrum/Scrum/tasks/the_daily_scrum_C8FB7908.html
Effort Estimation¶
Planning Poker¶
Requirements Management¶
Requirements Modeling¶
Agile Modeling¶
“Agile Modeling Best Practices”
http://agilemodeling.com/essays/bestPractices.htm
- Model storming: http://agilemodeling.com/essays/modelStorming.htm
- JIT: Just-in-time
- BRUF: “Big Requirements Up Front”
- JBGE: “Just Barely Good Enough” (/
anticipation
) - Executable Specifications: http://agilemodeling.com/essays/executableSpecifications.htm
- TST: Test Driven Development and Behavior Driven Development tests (Test Automation)
- BLD: Software Packaging
- BLD: Configuration Management (Infrastructure as Code)
- BLD: Continuous Integration (one-click build)
- BLD: Continuous Deployment (one-click deploy)
See also:
- Scrum, Extreme Programming, Unified Process
- Collaborative Software
- Six Patterns of Collaboration
Test Driven Development¶
Behavior Driven Development¶
Given-When-Then¶
- Given (context)
- When (event)
- Then (expect, should, assert)
Given When Then | Given | When | Then |
Hoare Logic | precondition P |
command C |
postcondition Q |
Terms | Input: State, Context (,RDF), Database | Input: Parametrized Event, Message | Output Conditions, Rules, Expressions, Assertions, Expectations |
HTTP 1.1 | app state (etc, fs, db) | request (URI(URN|URL))?attrs=values | response ({json: True}) |
Channels | app state (etc, fs, db) | event(message_in) | conditions(message_out) |
Vows | test suite, @Vows.Batch , Vows.Context , topic() |
vow – def should_quack(self, topic): |
vow – `` expect(topic.quack()).to_equal(value)`` |
User Story¶
A user story is a couple of simple sentences expressing desired competencies, features, or behaviors of a system.
Ideally, user stories can simply be transformed to e.g. TDD and BDD tests.
Example user stories:
- “[role] can [action [objects]] [in order to / so that [...]]”
- “Users can login and post Things with a description and a url”
- “Users can login and post Things with a description and a url in order to have something with a URL to share”
- “Users can share shared Things with a URL”
User story formats
Card, Conversation, Confirmation¶
The Three C’s of User Story Requirements Modeling are Card, Conversation, and Confirmation.
- Card
- Issue
name
anddescription
(ordescription__markdown_github
). - Codelabels
- Issue
- Conversation
- Issue comment thread or threads (with full URLs)
- Pull Request comment thread or threads (with full URLs)
- Codelabels
- Confirmation
- Issue
status
: open / closed - Issue Labels: { duplicate, wontftix }
- Issue labels: { ready, in progress, } TODO
- https://waffle.io/ uses ready, in progress labels to build an interactive Kanban board with columns from github issue labels.
- Issue
Use Case¶
A use case describes actions, actors, and goals.
- UML defines a formal model for Use cases
and use case diagrams.
- Use case diagrams may involve stick figures as symbols for actors.
- In practice, the use case is a more formal requirement model than the User Story.
- In terms of process workflow, a Use Cases could be derived from zero or more User Stories.
- There are many domain modeling tools for e.g. UML; each with various levels of support for round-trip between modeling tool and e.g. code that you add after transforming to which is sometimes referred to as stub code.
Requirements Traceability¶
-
issue numbers (GitHub, BitBucket, ):
- ad-hoc (the numbers probably won’t be sequential)
- issue
#1
: Mission, Project list - issue
#2
: Project - issue
#3
: Goal - issue
#4
: an Objective for goal#3
- With a fixed, preallocated numbering system,
this would be e.g.
project:3.1
; however, issue ticket numbers are sequential by order of creation- the numbers would then appear somewhat arbitrary
- there would be no need to renumber things
- With a fixed, preallocated numbering system,
this would be e.g.
#perma-link
document URL fragments#mission-statement
#project-<name>
/#p-<name>
#goal-<name>
/#g-<name>
#objective-<name>
/#o-<name>
implementations:
sphinx restructuredtext (with Ctrl-F’able explicit links):
.. _mission: ######### Mission ######### .. _project-name: *************** Project: Name *************** .. _goal-name: Goal: Name =============== .. _objective-name: Objective: Name ---------------- * [ ] Task: Name .. _task-name: Task: Name ^^^^^^^^^^^^
markdown:
# Mission: ## Project: Name ## Goal: Name ### Objective: Name * [ ] Task: Name #### Task: Name
- see also:
#pyglobalgoals
- https://github.com/schemaorg/schemaorg/issues/1127
- [ ]
schema:Project
- [ ]
schema:Goal
- [ ]
schema:Objective
- [ ]
schema:Task
- [ ]
- https://github.com/schemaorg/schemaorg/issues/1127
-
MISSION GOAL OBJECTIVE IDEA: idea STRY: user story -> TST, ENH, BUG, RLS ; -> CASE ; + REQ CASE: use case -> TST, ENH, BUG, RLS ; -> STRY ; + REQ REQ: requirement -> TST ; + REQ SPRINT: sprint == { REQ, CASE, STRY, TST, BUG, ENH, DOC, TST, RLS } TASK
Work Breakdown Structure¶
Mission¶
Project¶
- Specific
- Measurable
- Attainable
- Relevant
- Time-bound
Checkboxes¶
Checkboxes can be expressed in plaintext with square brackets
surrounding a space character, a state character, or a completion
character (e.g. lowercase x
or uppercase X
).
A Markdown list of items with checkboxes and Codelabels:
- [x] ENH: one **bold**
- [ ] ENH: two *italic*
- [x] TST: two.one
- [ ] DOC: two.two \* two
- [ ] RLS: ``three`` (also `three`)
A ReStructuredText list of items with checkboxes and Codelabels:
* [x] ENH: one **bold**
* [ ] ENH: two *italic*
* [x] TST: two.one
* [ ] DOC: two.two \* two
* [ ] RLS: ``three``
TODO
* [ ] Unchecked # github
* [x] Checked # github
* [X] Checked # github
- [X (YYYY-MM-DD HH:MM:SS)]
- [-]
- [o]
- [O]
- [[ ]] # text
- [[x]] # text
- [[o]] # text
- \\[ ] # text
- \\[-] # text
- \\[o] # text
- \\[x] # text
- \\[X] # text
Codelabels¶
Codelabels (code labels) are three-letter codes with which commit messages can be prefixed.
CODE Label color name background text
---- -------------- --------------- ---------- -------
BLD build light green #bfe5bf #2a332a
BUG bug red #fc2929 #ffffff (github default)
CLN cleanup light yellow #fef2c0 #333026
DOC documentation light blue #c7def8 #282d33
ENH enhancement blue #84b6eb #1c2733 (github default)
ETC config
PRF performance deep purple #5319e7 #ffffff
REF refactor dark green #009800 #ffffff
RLS release dark blue #0052cc #ffffff
SEC security orange #eb6420 #ffffff
TST test light purple #d4c5f9 #2b2833
UBY usability light pink #f7c6c7 #332829
DAT data
SCH schema
REQ requirement
ANN announcement
# Workflow Labels (e.g. for waffle.io kanban board columns)
ready dark sea green #006b75 #ffffff
in progress yellow #fbca04 #332900
# GitHub Labels
duplicate darker gray #cccccc #333333 (github default)
help wanted green #159818 #ffffff (github default)
invalid light gray #e6e6e6 #333333 (github default)
question fuschia #cc317c #ffffff (github default)
wontfix white #ffffff #333333 (github default)
Note: All of these color codes (except for fuschia)
are drawn from the default GitHub palette.
Note: There are 23 labels listed here.
Note
For examples with color swatches in alphabetical order, see https://github.com/westurner/dotfiles/labels
Codelabel Syntax¶
BLD: Makefile: default -> test
BUG: setup.py: fix console_entrypoints
CLN: .gitignore: add "'pattern'
DOC: index.rst: add readme.rst to TOC
ENH: app/app.py: print 'Hello World' (closes #1)
PRF,REF: app/app.py: factor out of a loop
REF: app/app.py: extract into class and @staticmethods
RLS: setup.py,VERSION.txt,app/app.py: app v0.0.1
SEC: app/app.py: pass lists/tuples to sarge.run (#2)
TST: Extend tests for #1
UBY: app/app.py: update _('strings')
DAT: data/filename.csvw.jsonld: added, updated
SCH: data/context.jsonld: added, updated
COMMA,DELIMITED, SET: of prefix labels
THR
- Three-characters (code labels)
Code labels are helpful for:
- DOC: working with issue tracking tagging and labeling systems
- DOC: aggregating changes into release logs (
HISTORY
) - DOC: correlating changes with requirements (https://en.wikipedia.org/wiki/Requirements_traceability) [Requirements Traceability]
Note
These code labels are adapted and extended from a previous version of the pandas/CONTRIBUTING.md
Technical Debt¶
Refactoring¶
- Test Driven Development and Continuous Integration are key to “Fearless Refactoring”.
- Refactoring can reduce Technical Debt.
- While syntactical eloquence and/or elegance in one or more languages can be learned through Code Golf, practical Refactoring is more about structural, architectural decisions which anticipate change.
Continuous Delivery¶
https://en.wikipedia.org/wiki/Application_lifecycle_management
US Digital Services Playbook
Revision Control¶
Revision control, or version control, is a change management strategy for a project repository.
- There is a better way than emailing which version
of what the attachment was named
before the re: subject line changed.
- folders
- file versions
- usernames, change logs, signatures
- A Revision Control System (RCS) or Version Control System (VCS) is a software solution for handling change management with one or more project repositories.
- Centralized Revision Control Systems (RCS, VCS):
- CVS, SVN
- Distributed Revision Control Systems (DVCS):
- Git, Mercurial, Bazaar
Distributed Version Control¶
Distributed Revision Control Systems (“DVCS”) are version control systems designed for backup redundancy and collaboration.
- http://betterexplained.com/articles/intro-to-distributed-version-control-illustrated/
- Git, Mercurial, Bazaar
- vcs (Git, HubFlow, Mercurial)
Bisection¶
Bisection is a method for determining which change causes a fault (or a specific test to change from passing to failing or vice-versa).
Many bisection algorithms take a start and end (“between here and here”)
and do a binary search (“this half or that half”);
checking out each revision and running a script that
should return 0
for OK, or non-zero.
Code bisection with git:
- https://www.kernel.org/pub/software/scm/git/docs/git-bisect.html
- https://www.kernel.org/pub/software/scm/git/docs/git-blame.html
- http://git-scm.com/book/en/Git-Tools-Debugging-with-Git
Code bisection with mercurial:
Branching & Merging¶
- http://www.infoq.com/articles/agile-version-control
- HubFlow – a Git release workflow
for GitHub and Pull Requests with always-stable
(“firm”)
master
anddevelop
branches, andhotfix/
,feature/
, andrelease/
branches; consistently woven.
Test Automation¶
- Test Driven Development – write tests for bugs, use cases, user stories, features, workflows, changes.
- Continuous Integration – always run the tests when checking code into Revision Control and before releasing.
- Continuous Delivery – test the application delivery and deployment (script a complete system install e.g. with virtualization like Docker).
Continuous Integration¶
- Test Automation
- Configuration Management
- #DevOps
Continuous Deployment¶
- Configuration Management
- Virtualization
- #DevOps