Fill and Sign the Informed Consent and Request for Colposcopy

A  System  for  Proac/ve,  Con/nuous,   and  Efficient  Collec/on  of  Digital   Evidence   Clay  Shields   Ophir  Frieder   Mark  Maloof   Department  of  Computer  Science   Georgetown  University   Mo/va/on   • How  do  you  find  Bradley   Manning?   (Assuming  Adrian  Lamo   doesn’t  turn  him  in)   – Very  large  network   – Some  documents  from  that   network   • Who  had  access?   • Who  released  them?   Scalable  Internal  Inves/ga/ons   • You  own  the  equipment  in  advance   – Can  plan  for  inves/ga/ons  in  advance   • However   – Forensic  tools  were  developed  for  situa/ons  where   equipment  was  seized   • Assume  no  prior  access  to  equipment   – Added  client-­‐server  model  allows  remote  inves/ga/on   • Some  agent  capabili/es   • Huge  amounts  of  informa/on  distributed  across  many   machines   – Where  have  we  seen  this?   PROOFS   An  Informa/on  Retrieval  Approach   • Google  for  forensic  examiners   – Save  informa/on  in  advance  to  make  life  easier  later   – Centralize  it  for  easy  searching   • Parse  and  record  data  about  file  contents   – When  files  are  unlinked  or  closed     – Store  informa/on  in  a  scalable  manner   • Allows  inves/ga/on  over  four  axes   – – – – Time   User  ID   System  ID   File  contents   Forensic  Document  Signatures   • The  informa/on  stored  is  a  document  signature   – Store  in  a  central  database  for  ease  of  searching   • Local  storage  temporarily  when  needed   – Metadata  about  the  file   • Path,  size,  owner,  MAC  dates   – One  or  more  file  fingerprints   • Computed  from  file  content   – Outside  In  or  similar  can  be  used  to  extract  text   • Unlike  hashes  can  match  across  edits   • Can  match  across  file  types   Fingerprint  Crea/on   • Use  a  training  set  of  documents   – Documents  that  are  similar  to  those  sought   – General  documents  in  correct  language   • Extract  sta/s/cally  important  terms   |#D| idfT = log 1+ | # DT | • Create  a  dic/onary  of  terms  within  a  range  of  IDFs   – Low  IDFs  too  common   – High  IDFs  too  dis/nct   Bit  Vector  Fingerprints   • A  Bit  Vector  fingerprint  shows  which  dic/onary   terms  were  present  in  a  document   – Process  document   – For  each  term  in  document  in  dic/onary,  mark  that   posi/on     • Generally  sparse,  highly  compressible   • Can  add  terms  to  end  of  vector  over  /me   – Allows  for  different  dic/onary  versions   • Robust  matching  using  cosine  similarity   – Parameter  allows  tradeoff  of  accuracy   Bit  Vector  Size  vs.  Performance   Precision   enron - Dictionary Size vs. Precision, various IDF ranges 1 Average Precision 0.8 0.6 0.4 1% Dictionary Sample 5% Dictionary Sample 10% Dictionary Sample 20% Dictionary Sample 30% Dictionary Sample 0.2 0 0 5000 10000 15000 20000 25000 30000 35000 40000 Average Dictionary Size (uncompressed bits) 45000 50000 55000 60000 Bit  Vector  Size  vs.  Performance   Recall   enron - Dictionary Size vs. Recall, various IDF ranges 1 Average Recall 0.8 0.6 0.4 1% Dictionary Sample 5% Dictionary Sample 10% Dictionary Sample 20% Dictionary Sample 30% Dictionary Sample 0.2 0 0 5000 10000 15000 20000 25000 30000 35000 40000 Average Dictionary Size (uncompressed bits) 45000 50000 55000 60000 Performance  with  Errors   • Forensic  recovery  oZen  finds  file  fragments   • Text  extrac/on  is  prone  to  errors   – Forma[ng   – OCR   • Simulate  these  errors  in  our  tes/ng   – Delete  sec/ons,  tokens,  characters   – Insert  tokens,  characters,  whitespace   – Change  token,  character   – Automated  edits  for  size   Bit  Vector  Fingerprint   Robustness   Enron Dataset, Matcher Setting 80, 95% CI Error Bars 1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Average Precision Average Recall 0 10 20 30 40 50 60 70 % of File Deleted Contiguously 80 90 100 Bit  Vector  Fingerprint   Robustness   Enron Dataset, Matcher Setting 60, 95% CI Error Bars 1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Average Precision Average Recall 0 10 20 30 40 50 60 70 % of File Deleted Contiguously 80 90 100 Bit  Vector  Fingerprint   Robustness   Enron Dataset, Matcher Setting 40, 95% CI Error Bars 1.1 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Average Precision Average Recall 0 10 20 30 40 50 60 70 % of File Deleted Contiguously 80 90 100 Overhead   • Two  concerns:     – Storage   – CPU  usage   • Trace  driven  simula/on  to  determine   feasability   – Email  traces  from  Georgetown  University   • ~8800  users   – SOS  file  system  traces  from  Harvard  server     • Older,  but  public   Email  Overhead     Emails Processed by Georgetown Mail Server, Dec. 2008 - Oct. 2009 900000 800000 Emails Received Emails Sent Number of Email 700000 600000 500000 400000 300000 200000 100000 0 01/01/09 03/01/09 05/01/09 Date 07/01/09 09/01/09 11/01/09 Email  Overhead   Cumulative Fingerprint Storage Required with 1,024 B Email Signatures 180000 Storage Space in MB 160000 140000 Received Email Fingerprint Storage Sent Email Fingerprint Storage Total Email Fingerprint Storage 120000 100000 80000 60000 40000 20000 0 01/01/09 03/01/09 05/01/09 Date 07/01/09 09/01/09 11/01/09 Server  Storage  Overhead   3000 200000 DEASNA Storage Space Files Removed Per Day 150000 2000 100000 1000 50000 0 10/19 10/26 11/02 Date 11/09 11/16 0 11/23 Daily Deleted Files Storage Space Required in MB Cumulative Storage Space for DEASNA with 1,024 B Signatures Server  Ac/vity   Server  CPU  Ac/vity   Server  CPU  Ac/vity   Supported  Inves/ga/ons   • Leaked  Documents   – Given  a  recovered  document,  find  all  users  that  have  ever  held  a  copy   • Misuse  Inves/ga/ons   – Determine  what  files  an  employee  was  copying  or  accessing   – Determine  email  correspondence,  web  access   • Keyword  Search  Overwriaen  Files   – Iden/fy  which  systems  to  preserve  when  so  required   • Intrusion  Response   – Given  a  file  that  was  used  in  an  intrusion,  find  all  systems  that  had  that   file   • Examina/on  support   – Iden/fy  fragment  sources   • Lost  equipment  review   – What  was  on  that  laptop  leZ  in  the  taxi?   Con/nuing  Work   • Use  fingerprints  as  an  alterna/ve  to  hashes  in   large  data  sets   – Fingerprint  documents  by  sec/on   • OS  Hooks   – Process  files  as  they  are  modified  or  deleted   • Fast  fingerprint  matching   – Cosine  matching  is  not  suitable  for  Bloom  Filters   • Create  signatures  for  non-­‐text  files   – Images,  audio,  video,  executables,  source  code   Summary   • PROOFS  allows  for  efficient  proac/ve   collec/on   – Google  for  forensic  examiners   – Make  inves/ga/ons  faster,  cheaper  and  more   accurate   • Fingerprints  have  other  uses  as  well   – Recognizing  files  in  large  data  sets   A  System  for  Proac/ve,  Con/nuous,   and  Efficient  Collec/on  of  Digital   Evidence   Clay  Shields   Ophir  Frieder   Mark  Maloof   Department  of  Computer  Science   Georgetown  University