Peter B. Lubell-Doughtie

Improving Result Diversity using Probabilistic Latent Semantic Indexing

noreply@blogger.com (lubell-doughtie) — Tue, 08 Jun 2010 15:29:00 +0000

ABSTRACT
The diversity of a set of search results reflects that set’s coverage of multiple interpretations of the query it is based upon. Interest in search result diversity has increased as search engines struggle to return relevant information for ambiguous queries from an increasingly large data set. We develop a result diversification system which uses probabilistic latent semantic indexing to form clusters that are then used to reorder search results and increase result list diversity scores. We show that adjusting the influence of rank in the reordering algorithms improves their performance by finding a balance between the importance of rank and the importance of generated clusters. Finally, by applying the result diversification system to the known clusters used in judging, we generate reordered lists that estimate upper- bounds of the diversity scores.

Written for Advanced Information Retrieval taught by Marteen de Rijke.
Download the full paper [PDF].

Along the IJmeer

noreply@blogger.com (lubell-doughtie) — Mon, 22 Mar 2010 20:21:00 +0000

Helioid Preview

noreply@blogger.com (lubell-doughtie) — Wed, 17 Mar 2010 23:21:00 +0000

A couple months ago we opened up the Helioid development version to the public, but haven't yet announced it. There are significant performance problems stemming partly from limited server resources and partly from core algorithms being written in Ruby, which can't perform as a fast as a compiled language, such as C. There is a lot to still be done, but we believe this reflects a healthy starting point.

Please let us know if there are any suggestions, comments, criticisms, complaints, anything. The Helioid preview is accessible here: http://dev.helioid.com.

Main Loop

noreply@blogger.com (lubell-doughtie) — Sat, 05 Dec 2009 21:55:00 +0000

November 8th, 2006

Beneath these intentions, these tendencies, actions and reasons lies a complex tangle of biochemical processes, neurocomputational methods, quantum mechanical properties – or whatever the current explanatory fad happens to be. I persist in attempting to convince myself that I – my self, my conscious, my will – am in control, am authoritative, and maintain ascendancy. However the more effort I put into this belief the more obvious it becomes that it is untenable or at least paradoxical.

“But according to whose logic?” One may ask. Quickly I respond: “why to the only relevant logic, my logic.” Just as quickly it dawns upon me that I have once again fallen into the trap. For if we suppose that I am not in control of my reality and further suppose the supposition that I am in control of my reality is illogical it appears inconsistent to hold that “my logic” is the relevant logic when my logic is not “mine” at all but, from our first presumption, must be the logic of the reality imposed on me. This notion, this imposition of a reality, is this at all sensible or has there been some misstep taken in the core of my thinking to lead to such a contradictory statement? Although, perhaps it is not contradictory. Perhaps a clarification of terms is necessary. My reality is the world and the things as it and I perceive them. It is inescapable, it is ever present, and it is apparently poetic. Would reality be any less imposed upon me if I were a brain in a vat being fed sensory stimuli by some sort of machinery? It seems that it is a question of influence. How much influence can I have over the reality, how much interaction do I have with this reality?

The question then is not what is real but why is it real?

[December 5th, 2009 - I've now decided the question doesn't exist, can't be asked, or doesn't matter]

Jannette Pulido's Artwork

noreply@blogger.com (lubell-doughtie) — Tue, 01 Dec 2009 00:02:00 +0000

African Cosmologies - Tabwa

noreply@blogger.com (lubell-doughtie) — Wed, 25 Nov 2009 18:03:00 +0000

The summer of 2006 I worked with Manuel Jordan Perez researching the cosmological systems of civilizations in Africa. I have copied below text and diagrams from research done on the Tabwa civilization. My best to Manuel who led me in this research. Below is a chart of Tabwa cosmology and cosmogony:

TABWA COSMOLOGY

Central Ideas

1. Duality – most things have a positive and negative side to them these opposing sides are synthesized into a representative whole that is the manifest of the thing.

a. Moon

i. Positively may provide light and allow for safety, knowledge, and order to be perceived out of chaos and obscurity.

ii. Negatively may maliciously withhold its light.

iii. Goes through cycles but always containing within it positive and negative aspects

b. Chiefs

i. Offer knowledge and wisdom allowing people to perceive and defend themselves against danger.

ii. May be responsible for allowing evil to exist, example “Drunken King” Nkongolo Mwamba a tyrant and king of excess embodied by the sun.

iii. Must exist balancing positive and negative forces and events

1. Use lunar symbolism such as the Mpande shell to represent their success at this balancing.

c. Spirits

i. Mipasi

1. May be identified by an ng’anga as the source of an affliction in which case mikisi will be commissioned to represented the mipasi

2. May provide guidance through dreams serving as a source of wisdom and inspiration

ii. Mikisi

1. When ignored they gain attention through “-kisa” (the root verb) which means, “to cause suffering or bring misfortune”

2. “-kisa” also refers to the power that knowledge and the recognition of knowledge lends one.

3. These figures therefore represent the duality of the spirits effecting them.

iii. Mutumbi

1. Nearly blinded by daylight so rarely seen and apparently “invisible”.

2. Yet lots of tracks seen so must exist.

3. Similar to moon it this “invisible” but existent fashion.

iv. Twins

1. Take lunar names and are covered in ashes hiding their differences from each other when the new moon has risen.

2. Celebrated and given gifts in honor of their positive side.

3. Beaten with a switch and feared because of their negative side: the possibility that they could use their powers for evil.

2. Mulalambo

a. Mutumbi’s Tunnels

i. Mutumbi followed his dogs down a “seemingly endless tunnel” to the find the supreme deity, Leza, in the city of light.

b. Body Midline

i. Defines the symmetry and opposition existent in the human condition.
ii. There exist half-beings that represent cases of unresolved litigation in need of arbitration and the deployment of social justice.
iii. Myth of great leader associated with hunting whose body split along the midline and is composed of half beeswax and half flesh.

iv. This recognition of division symbolizes the unification of the body

c. Natural Formations

i. Mwila divide
ii. Lake Tanganyika
iii. Milkyway

Duality and Symmetry

Everyone and everything has power. It appears as though they are both made up of precisely the same parts just being viewed from different perspectives.

Below is a chart of Tabwa cosmology as based on opposition:

Below is a chart of the Balamwezi cosmology integrated into a Balamwezi pattern:

Below is a chart of the Balamwezi cosmology:

Structure (or lack there of)

noreply@blogger.com (lubell-doughtie) — Thu, 19 Nov 2009 11:37:00 +0000

Structure (~~or lack there of~~)

September 5th, 2008

Throughout the slowly decaying cortices of our humble: understanding abounds, knowledge surmises, meaning exists.

Shadows, claimed. Nothing more, still! We create? Still, levied over restlessness, challenged to remove, to “recompose” the, an original. An? True? Still must this not hear left false to the failure of transcription. For, it does did not rise there. It was, were it lay, stayed it mopped up without clarity, cleanliness. That, it has gone for to, realism. There is none, still. The access, entrance, possibility, destroyed through pliability. Left still back once, through creatibility. Back once more, through possibility. There back will always be somewhere more to go, right there forward, still rift the similarity, identically. Then still! Of what interest have we more to? For truly, grammatically, it was erased best, the end will be something only to be spoken of for when or if or past it has arisen then more, back will be for a turning point in the foundation. The expressability through it’s destruction can reach its. So then is attempted, a limitation on the limits? Placed by, the, there is one place that it can cannot. Arisen limits, imposed, created, irrelevant and still irrelevant. Usage lost to users importantly the unavoidable inversion.

October 25th, 2008

Then the discord, concealed so effectively tightly, would see light only through its inherent obscurity.

Is there a deeper truth to this? Is there some “fundamental framework” upon which we are building, out of which our lives – our minds – are created? Suppose the existence of a framework is as chimerical as a blue sky. Between the grimaces of oppressive skyscrapers our blue sky is not more than light shifting spectrum as it moves through humidity in our atmosphere, is not more than water modifying the wavelengths of photons. Ad infinitum we learn our precious sky is farther and farther from our comprehension. And soon that slips away as well, our comprehension, which we had been able to use so fluently to instruct us on our lack of comprehension, fails us. And finally a statement we could comprehend, that we cannot comprehend our lack of comprehension, is lost forever. But the framework, despite our tranquilization by the recursive, does it, hiding from the unlikely afflatus, remain? Would a proof, or a consensus by our world’s comparative literature departments, as to the frameworks impossibility have any effect on its existence or nonexistence?

Would the existence of this superstructure impose rules upon what can be built and the manner in which it can be built? Supposing that the answer to this question is yes, or at the least “maybe”, then there would exist an incentive to search for this truth or these truths, to discover the nature of this framework and investigate its properties so that we may better understand ourselves and our world. I inherently assume that the nature is hidden, that it must be “discovered”. (I additionally assume, or at the least heavily imply, the standard western distinction between being and existence.)

Would a simple existentialist standpoint solve all our problems? It could claim that the truths that need to be “discovered” and the “framework” underlying our world are inventions we have imposed upon the world and upon ourselves. The “deeper truth”, if I dare keep to this terminology, is that there is no deeper truth (or fundamental truth, or ultimate truth). The world is at it appears, the answers we search and yearn for are (1) non-existent or (2) answered by this statement. The second option seems entirely unsatisfying and is problematic in its circularity and that it is facially self-eliminative, yet deeply not. The first option begs the question, if the deeper truth’s non-existence answers the question of what is the deeper truth then the deeper truth does indeed have an existence, that of non-existence. And, quite directly, we are led to the transconsistent or: the same circularity of the second option.

[original with typos removed]

Environmental Naturalism

noreply@blogger.com (lubell-doughtie) — Tue, 17 Nov 2009 12:37:00 +0000

What is natural seems entirely dependent upon one's perception of one's environment. To understand one's environment one must understand one's self. This is a matter of emotional and mental confidence, a synchrony, a symphony.

A bonding between environment and self. Freedom from the self is freedom from that which is around you, an impossibility. The impossibility of freedom, from the self and one's environment, implies that self freedom comes from acceptance of one's mental and emotional self, as perceived through one's environment. From acceptance of one's environment, of one's surroundings and one's self.

This is a removal of the self through an understanding of nothingness or wholeness, and the inverse.
Mechanically, this implies environmental machinism: the belief that the natural world, in all it's extent is mechanically understandable. This is an impossibility, a confusion. To move beyond this confusion one must return to towards one's environment, towards one's self.

The Idea of a P2P Telecom

noreply@blogger.com (lubell-doughtie) — Mon, 09 Nov 2009 23:58:00 +0000

Below I present the idea for a P2P Telecom. That is, a telecom in which traffic is routed from sender to receiver through peers. Not through a central hub controlled by a company.

A few cons against this implementation.

you will only benefit from a P2P telecom if the people you're communicating with are using the P2P telecom.

Some many pros in favor of a P2P Telecom.

Since users will be controlling the network, users will be setting the prices and it will benefit all users for prices to be as low as possible. This is a stark contrast to the current system where prices are set by a handful of powerful companies with little worry of losing subscribers.

No additional infrastructure will need to be put in place. In fact, towers, antennas, relays, etc. can be removed from densely populated areas because they are unnecessary for a P2P Telecom to function.

A P2P Telecom will have more privacy because you will trust no one but yourself and encryption levels you control. If you want to use a theoretically unbreakable but data intensive encryption protocol that's up to you and no one will know your data but you and the other user you are communicating with. Like any communications system, the privacy of a P2P Telecom is not perfect but, as any cryptographer worthy of their title will tell you, the system is open and this is leaps and bounds more secure than the closed systems of big industry telecoms.

There is no centralized control. This helps to enable better data security and increased privacy, among other things. Without centralized control no one will be able to shut off the network and there will be no data overloads in case of an emergency (such as happened in New York City during September 11, 2001).
There will be increased capacity. This lack of centralized control enables increased capacity and redundancy in populated areas beyond anything approachable by current industry telecoms.

A P2P Telecom can transfer any form of data. Because of their popularity phone calls are the most obvious forms of data to be transferred but if desired the network can transfer anything else such as video or raw data.

It's clear to me that the pros of a P2P Telecom definitively outweigh the cons and it is time we start building this network, which turns out to be rather simple.

The Implementation of a P2P Telecom

The implementation is based on modifying the functionality of existing cell phones be they GSM, CDMA, or some other unheard of system.

Cell phones must be retrofit so they can act as peers with one another. At their core cell phones send and receive signals, these phones must be modified so that they can receive and send signals directly from and to other phones.

Accomplishing this may or may not require hardware modification. If hardware modification is required we must determine how this can be done. One simple method would be the use of a modified SIM chip. If the change needed is more fundamental a systematic approach to performing it can be developed.

Software modification will be required. We must:
1. Change what type of signals the phone looks for so that it can find signals from other phones
2. change how the phone interprets the signals it finds so that it can interpret them as:
  - Data passing. The phone must know how to move data from itself to other peers. This will require an algorithm informing the phone of the priorities for where to forward data it receives. This algorithm will benefit from, but not require, partial location awareness of peers and unique identification of peers. This can all be done while maintaining anonymity.
  - Data receiving. The phone must know that this data is destined for it and know how to handle decoding of the data.
  - Data sending. The phone must be able to uniquely encoded it's data so it is only decodable by the receiver.

The Long Distance Uplinks in a P2P Telecom

A problem with the described P2P telecom is that if you want to call someone outside of your peer neighborhoods range or someone who doesn't use the P2P system you'll have to place the call over the existing industry telecom network. A solution to this problem is to give every P2P neighborhood an internet uplink and connect multiple neighborhoods with a VoIP services. In this manner, if either of the above situations occur, the call can be placed using VoIP technologies.

The number of peers with access to the uplink and therefore the bandwidth of the circuit between the peers and the uplink will depend on the volume of calls going out of the local neighborhood.

The Algorithms for a P2P Telecom

Finding a peer and establishing a circuit (Draft finished)

Sending data through a circuit - Encrypt data using public/private key system

Throttling of uplink bandwidth - Better to deny calls that increase latency on calls

Location anonymnity

Images from the early 00s

noreply@blogger.com (lubell-doughtie) — Sun, 08 Nov 2009 22:47:00 +0000

Information Security Bookmarks

noreply@blogger.com (lubell-doughtie) — Sat, 07 Nov 2009 11:39:00 +0000

Below is a list of bookmarks dealing with information security. The list hasn't been thoroughly reviewed so please let me know if something is out of date and should be removed. One of the most helpful sites to me when I began researching info sec was a blog post with a bunch of bookmarks, hopefully this will be similarly helpful to you.

Information Security Companies Antivirus, anti-spyware, anti-spam, firewall. Protect yourself with Panda Security Beyond Security - Automated Vulnerability Assessment and Management CSC: Consulting, Systems Integration and Outsourcing Damballa > - The Front Line Against Bot Armies Digital Armaments for Intelligence Dreamlab Technologies eEye Digital Security » Internet Security | Vulnerability Assessment | Computer Network Security | Vulnerability Management<BR> esentire online security FindNot - Anonymous Surfing, Anonymous Email & Anonymous Internet Flylogic Engineering FYRM Associates, Inc. - Tools GFI - Fax server, Exchange anti-virus, anti-spam, vulnerability management and email archiving software HBGary Linux Internet/Network Security Software, Hardware, & Services | Guardian Digital, Inc. MainNerve.com - Securing Critical Systems Marshal :: Threat Research and Content Engineering (TRACE) News and Events - [WhiteHat Security, Inc.] (Web Application Security) SCRT - Information Security - Home Securify. Monitor and Verify Who, What & Where on Your Network. Security Intelligence Technology Group : Cellular Intercept Systems and Equipment Security-Assessment.com: Advise - Assess - Assure SolarWinds: Network Management Software – Designed by Network Engineers for Network Engineers Sûnnet Beskerming - Commentary & Insight From Sûnnet Beskerming Sysdream / Sécurité Informatique :: Home Welcome to U0vd Security Corporation Home - RSA, The Security Division of EMC ID QUANTIQUE

Information Security Research Authentication Kerberos Kerberos: The Network Authentication Protocol Radius An Analysis of the RADIUS Authentication Protocol FreeRADIUS: The world's most popular RADIUS Server Blogs A surreptitious look over the work of an innovative startup — Subreption blog Aviv Raff On .NET Black Security | Main Blog - Research - SecureWorks Chris Shiflett: PHP and Web Application Security Computer Security Research - McAfee Avert Labs Blog call dwerd ptr [6c756973] Darknet - The Darkside | Ethical Hacking, Penetration Testing & Computer Security Determina Security Blog Devels playground DShield; Cooperative Network Security Community - Internet Security - dshield F-Secure Weblog : News from the Lab Google Online Security Blog Hack a Day Hackszine.com Home ha.ckers.org web application security lab hiredhacker.com - relevant ramblings of an ethical hacker invisiblethings' blog Jeremiah Grossman Linux Security, Single Packet Authorization, Intrusion Detection, iptables Log Analysis Liquidmatrix Security Digest leetupload.com Matasano Chargen Over The Wire - PullThePlug Panda Research Blog, leading the way in proactive malware detection PaulDotCom's Web Site: Security Archives PLDsoft ::: IT-Security, Coding,Programming, Security Videos, HackITs, Web Security Challenge System - Home Prevx Blog RETURN $ecure; Security Vulnerability Research & Defense System Advancements at the Monastery Tenable Network Security The Art of Software Security Assessment The Turkey Curse TrendLabs | Malware Blog - by Trend Micro tssci security Uninformed ush.it - a beautiful place Websense Security Labs Blog Silent noise | - about spam, trojans and other nasty stuff Network Security Research and AI 33 Bits of Entropy Silviocesare’s Weblog Renesys Blog Cards Proximity Cards Home - PCI Security Standards Council CD & DVD SlySoft Forum Communities .:[ packet storm ]:. - http://packetstormsecurity.org/ Electronic Frontier Foundation | Defending Freedom in the Digital World Expert Tips to Deal with Hacking Hack In The Box :: Keeping Knowledge Free Home - The Community's Center for Security johnny.ihackstuff.com - Home Network security articles and hacking prevention resources for the government and general public. Covering all aspects of Computer Hacking, including tutorials and exploit downloads. NIST.gov - Computer Security Division - Computer Security Resource Center Ryan1918 DOT COM :: Index Thwack totse.com | Technology SecDev.org Grand Idea Studio Welcome to Cryptography Research h4cky0u • The #1 hacking, security and infosec forum Security Xploded by Nagareshwar Talekar NCHOVY 인터넷 스톰 센터 : 홈 SecurityTube - Watch, Learn and Contribute Computer Security Videos Malware Forge darklab.org Darklab consists of a number of people, that have formed a group to share knowledge and interesting informations about computer security and IT in general. Remote Exploit Forums - Home of BackTrack Backtrack Remote-Exploit Forums Sécurité informatique et veille en sécurité informatique Site de veille en sécurité informatique: derniers bulletins de sécurité informatique traduits en français, tutoriaux sur la sécurité informatique et le hacking, logiciels de sécurité internet... Denial of Service Denial of Service Attacks - DDOS, SMURF, FRAGGLE, TRINOO DoS, DDoS, denial of service, distributed denial of service attack, icmp, smurf, http flood, dos attack, ddos attack Documentation Magazines .:: Phrack Magazine ::. 0x000000 - THE HACKER WEBZINE 2600: The Hacker Quarterly 7A69XG : b4b0.org - zine : :: The Bug! Magazine - Information wants to be free :: Binary Revolution - The Revolution will be Digitized! bUTCHERED fR0M iNSiDE - Security Magazine Die Datenschleuder Magazine MISC - Edition française Black Hat : Archives Index Black Hat Multimedia - Presentation, Audio and Video Archives bitsavers.org Computer System and Network Security Home page HydraFlux - DeepRants INFO-SEC dot EU - Information Security, Tools, Programming and Papers Index of /e-books L0T3K, Digital Girlie Juice Main Page - DocDroppers Wiretapped - Computer Security Software etc. Filtering Advisories : Bypassing content filtering - security vulnerabilities database Firewalls ipfw howto Forensics Forensic Acquisition Utilities Forensic Analysis of a Live Linux System, Pt. 1 NIST Computer Forensic Tool Testing Program Secure Deletion of Data from Magnetic and Solid-State Memory JDAFTS Forums sla.ckers.org web application security forum Blackhat-forums- Hacking / IT Security Community Iran Hackerz Security Team [Habo] | Startseite Hardening BASTILLE-LINUX code jmorris Security-Enhanced Linux Hardware cq.cx OpenCiphers The Hardware Book - HwB limor Wii Menu 4.2: a lack of imagination Infrastructure Critical Infrastructure Protection Locks Security.Org - Investigative Law Offices January 22, 2008 TOOOL Waag Society - TOOOL Lockpick workshops Mac OS Mac OS X Hacks MrDehbuga Home Technical Note TN2124: Mac OS X Debugging Magic Mac Security Hacking Apple OSX SYMWorkshop - Metalab The Mac Security Blog Malware Botnets BotHunter Free Internet Distribution Page TrustedSource - Storm Tracker Lowkeysoft Publications: Proxy Botnet Analysis Shadowserver Foundation - Main - HomePage Rootkits Anti Rootkit Software, News, Articles and Forums bluepillproject.org chkrootkit -- locally checks for signs of a rootkit invisiblethings.org rootkit.com Stealth MBR rootkit Virus 29A Labs Above the Haze-Clouds Cryptovirology Viruslist.com - Information About Viruses, Hackers and Spam VirusTotal - Free Online Virus and Malware Scan Welcome! (VX heavens) News Rootsecure.net | Home SecurityFocus Zone-H.org - IT Security Information Network TorrentFreak | The Latest Torrent News and Filesharing Info ...And you will know me by the trail of bits Security Absurdity sudosecure.net Malware Database Operations Legal Summary of the Provisions of the Sarbanes-Oxley Act of 2002 COSO Standards ISO27k infosec management standards ISF The Standard Of Good Practice ITIL ®- IT Infrastructure Library Organizations Italia +Ma's Reversing - main page A/I :: Home ALCEI Codex Alpe Adria 2007 (0xAA.org) - Digital Culture between Alps and Beaches FreakNet Medialab, Catania HackIt 0x0B - 26-27-28 settembre - Palermo Home - UIC Horde Demoparty IKKISOFT - Ethical Hacking, Web Application Security, Digital Freedom Isole nella Rete kill-9.it Metro Olografix s0ftpr0ject spaghettiphreakers.tk Winnie lives here / Winnie vive qui www.sikurezza.org - Italian Security Mailing List | Main / Home Antifork Research, Inc. Big Brother Is Watching You CPNI Cutting-edge Think tank | GNUCITIZEN cDc -+- COWFEED -+- cDc DoxPara Research Droids Corporation eicar - Home Grand Idea Studio grsecurity Hacker Factor: Computer Consulting, Solutions, and Security Hackers For Charity.ORG Hackers Profiling Project Hacking Truths Hacktivismo: News Horst Görtz Institute for IT-Security http://www.devhell.org/ Intruders Tiger Team Security Irongeek.com KD-Team - Home LASEC New York City 2600 Ninja Strike Force NoWires Research Group nologin -- o0o.nu security research [ news ] PHENOELIT Recurity Labs RedTeam Remote-Exploit.org - Supplying offensive security products to the world Roothack.org Rstack.org rezen.org SecuObs.com - L'observatoire de la sécurité internet Security Practices and Research Student Association Security-Protocols :: Computer Security Research Shadowserver Foundation - Main - HomePage SRI Malware Threat Center trifinite.org - the home of the trifinite.group Wehnus Welcome to DEFCON®, the Largest Underground Hacking Convention in the World Wisec - The WIse SECurity [FM] == FelineMenace [ISR] Infobyte Security Research Homepage of PaX Remote-Exploit.org - Supplying offensive security products to the world Remote Exploit Security Group. Offensive Security Training. Home of BackTrack, a Live CD for Security Penetration Testing. Reconnaissance American Civil Liberties Union httprecon project Internet Archive Recurity Labs PortBunny SITE Intelligence Group The Art of Electronic Deduction - DocDroppers Unicornscan [DP] - Default DB Ports, Computer Ports, TCP/UDP Ports, Services, Protocols [the new p0f] Satellite Feedhunter Rini dvbsnoop - DVB Stream Analyzer, MPEG Analyzer dvbsnoop - dvbsnoop is a dvb stream analyzer and monitoring tool for DVB data transmission streams and related data streams. MPEG, MHP, DSM-CC, TS, PES PY2ADN - Ham Radio (Adinei Brochi) UHF-Satcom.com - The #1 online resource for VHF to EHF satellite reception and monitoring UHF-Satcom.com - the online place for VHF to EHF satcom monitoring uhf vhf shf ehf ku-band c-band p-band l-band s-band x-band PY2BBS - Hamradio Page Side Channel SCA Lab Tools Analysis NetSA Security Suite Analysis Manifold HiPPIE NetworkMiner - Passive Sniffer & Packet Analysis Tool for Windows | Darknet - The Darkside Goolag Wepawet » Home Anonymnity https://bridges.torproject.org/ Tor: anonymity online OnionCat – An IP-Transparent Tor Hidden Service Connector TorCheck at Xenobite.eu - HTTP-Mode XROXY.COM - more than just proxy XROXY.COM - Site offers free access to the frequently updated database of open proxies via flexible user interface allowing selection of proxy properties like its type, port number, connection latency, etc. Japanese Proxy Server Feel free to browse the internet at school with YourProxy.com to unblock websites like Myspace, Bebo, Facebook, Friendster, hi5 and more! Designed by FreeProxyTemplates.com PolySolve will help you with your math related needs with our powerful cgi applications Captcha captcha bypass captcha OCR Verification image bypass captcha decoder captcha reader PWNtcha – caca labs CAPTCHA bypass. Cheap and easy CAPTCHA solving desc CAPTCHA.ru Open Source TCPDUMP public repository websnarf - HTTP Port Listener/Capture Daemon raWPacket » HeX packet-o-matic : the real time packet processor Database Top 15 free SQL Injection Scanners - Security-Hacks.com Exploitation The Metasploit Project openssl-too-open ShellForge Injection sqlninja - a SQL Server injection & takeover tool squeeza iKAT - Interactive Kiosk Attack Tool - paul <at> ha.cked.net Durzosploit Introduction - Engineering For Fun Main Page - Engineering For Fun KON-BOOT - ULTIMATE WINDOWS/LINUX HACKING UTILITY :-) a little bit of security Fuzzing Fuzzing Software Tools // iDefense Labs AxMan ActiveX Fuzzer Edge-Security - Wfuzz IDS libemu - shellcode detection Packers SecureReality Remote Poison Ivy RAT reDuh - TCP Redirection over HTTP HSC - Tools - Dns2tcp Trace Insecure.Org - Nmap Free Security Scanner, Tools & Hacking resources tcptraceroute development COTSE-Steganography Tools CryoPID - A Process Freezer for Linux ettercap girardin online tools HT Editor Index of ftp://ftp.ee.lbl.gov/ Mercurial repositories index Online malware scan shimmer Tools | iSEC Partners Index of /code/skape PktAnon - Packet trace anonymization SecuriTeam™ - ArpON - ARP Management System Category:Tools - Insurgency Wiki VoIP VoIP Hopper...Jumping from one VLAN to the next! VOIPSA : Resources : VoIP Security Tools UNIStimpy - esentire online security TacVoIP: Hardcore VoIP Security WarVOX: Introduction Vulnerabilities Luigi Auriemma milw0rm - exploits : vulnerabilities : videos : papers : shellcode National Vulnerability Database Home NEOHAPSIS - Peace of Mind Through Integrity and Insight OSVDB: The Open Source Vulnerability Database TrustedSource - Home URI Use and Abuse: An Unpatched Vulnerability in Web Browsers and Applications | Project Afterlight. Articles, News, Updates, and Reviews on Technology and Life. Vulnerability and Virus Information - Secunia VulnWatch: vulnerability disclosure list Wabisabilabi Vulnerabilities Virtualization SecuriTeam™ - Path Traversal Vulnerability in VMware's Shared Folders Implementation OpenVAS - OpenVAS - Open Vulnerability Assessment System Community Site War Games The Happy Hacker -- Computers that you can legally break into! Web BrowserFry™ The Spanner - A tool for designers dealing with programmers dealing with designers... PhishTank | Join the fight against phishing ncache - Google Code Brainspl.at XSSed - XSS (cross-site scripting) information and vulnerable websites archive TwitPwn Wireless Bluetooth Bluetooth Penetration Testing Framework Cell Phones Cell phone jammer, SH066P Handheld mobile phone blocker Free Nokia Unlock Code Generator by GSM Liberty Nokiafree unlock phone codes calculator software UnlockNokia.US: Nokia (GSM) Unlocking. www.lur.nu - Free mobile content and unlocking. Get your java games and ringtones for free Welcome to epocsoft CSI Stick - Cell Phone Data Extraction blog.iphone-dev.org NeoPwn Mobile Pentesting - First Ever Network Auditing Distribution for a Mobile Phone Platform HaRET - Handhelds.org MoinMoin Wiki GSM GSM-Forum gsm - THC Wiki GSM Interception RFID RFIDIOt.org - RFID IO tools WiFi KARMA Attacks Radioed Machines Automatically KisMAC Projects - fakeAP rfakeap homepage Softmac - Trac kismacng – Trac XTest:Readme Main [Aircrack-ng] Crack-wpa.fr Accueil, Votre portail wifi, réseaux et sécurité. Forum, tutoriels, Backtrack... Bienvenue sur crack-wpa.fr, pour tout ce qui concerne le crack WEP et le crack WPA c'est ici que ça se passe. Goatse at Defcon -- brought to you by airpwn Wave Bubble

VoIP Watermarking Defenses

noreply@blogger.com (lubell-doughtie) — Wed, 20 Feb 2008 09:15:00 +0000

A couple of years ago (2005) the Rome Air Force Base sponsored research [1] into de-anonymizing VoIP traffic. The researchers developed a modification to the Linux Kernel which inserted a watermark into Skype VoIP traffic that is passed through a low-latency anonymizing network. A 24-bit watermark is inserted through the modulation of the inter-packet timing of data packets. This is essentially the establishment of a covert channel through a timing attack.

The attacker reads the probabilistically hidden bits in the traffic to reconstruct and identify of the originating and terminating nodes of a VoIP call. A defense against this would be to scrub your outgoing traffic to remove the covert channel or increase the probability of error in bit recovery beyond the acceptable rate. The attacker is not manipulating packets as they leave the origin, since then they would presumably already know the origin. The suggested implementation is to watermark packets as they transit through a VoIP gateway. Because of this it is necessary to scrub packets beyond the gateway; after they have been marked.

More interesting would be to alter the packet timing in a controlled manner and embed bits of your choosing. If you had enough knowledge as to how bit patterns are assigned to identities you could arbitrarily alter your identity and pose as another. You could also add incorrect watermarks to random VoIP traffic.

To detect a watermark you can exploit the embedding process. The technique relies on existing latency in the VoIP calls and is able to function with around 20ms - 30ms of latency by making a 3ms adjustment to packet arrival times. A suggestion is to make the latency as low as possible therefore making the existence of a watermark more detectable since the latency would need to be adjusted to unexpected levels. It may not be feasible to keep low latency for a long period of time but that would not necessarily be necessary. Latency could intermittently be pushed to the lowest possible levels and a check for embedded bits could be performed. The method uses the existing latency in the first minutes of the call to determine what an acceptable level of latency to add is. Exploiting this, the first minutes (or so) of the call could be made with high, but still believable, latency so the attacker embeds bits with the appropriate higher latency. Once a watermark has been embedded the latency could be significantly reduced and the alteration of packet timing should be noticeable.

Covert channels based on packet timing have many applications, beyond de-anonymization, and could be made very difficult to detect. Steganographic style embedding of traffic is a possibility as well as watermarking for authentication purposes by the originating and terminating parties.

[1] S. Chen, S. Jajodia, and X. Wang. Tracking Anonymous Peer-to-Peer VoIP Calls on the Internet. In CCS '05. ACM, November 2005.

Using Video as a Covert Channel

noreply@blogger.com (lubell-doughtie) — Tue, 19 Feb 2008 09:10:00 +0000

With steganography being all the rage these days why not up the ante and use video as a covert channel? (I haven't looked around for this but I bet someone else is already doing it.) With all the bandwidth, and the data already being put through some sort of transcoding, the opportunity to embed seems ripe. I'm not sure how this would work, but let's assume the process is: (a) hardware camera, (b) software transcoding and compression of signal, (c) network traversal, (d) software transcoding and decompression of data, (e) video display. We'll assume adding data before (b) and extracting data after (d) will not work due to loss from the transcoding. To do live video stego insert a step between (b) and (c) to add data and between (c) and (d) to extract data. The method would be rather detectable. A better solution may be to drown yourself in transcoding and (de)compression code for a while and develop a stego (or stego capable) codec or a stego module for either a specific existing codec or any codec satisfying certain requirements. (Do these exist alraedy?)

Embedding of small amounts of data should be difficult to detect. A simple (well, maybe) exercise would be to embed a chat in a video conference. Binary files, VoIP, other video conferences, multimedia files, or any sort of network traffic are all nice things to be able to embed.

The hardest needle to find is the needle you're not looking for.

(Relatedly, it would be desirable to be able to determine if someone has found your covert channel. How or if this could be done using lacking some sort of quantum crypto I'm not sure. Use of an additional channel to influence the convert channel in ways known by the covert parties might work but not be provably correct. The idea would be that when an ease dropper is present their would be some impediment to making the expected change in the channel, similar to quantum crypto. Although, "some impediment" is pointlessly vague.)

Oh, It's a "Friendly" Unrequested Illegal Malware Intrusion

noreply@blogger.com (lubell-doughtie) — Mon, 18 Feb 2008 02:56:00 +0000

Microsoft, in their quest to rid the world of the poorly written code they have burdened it with, has decided that exploiting vulnerabilities in their products in order to repair their products is an acceptable practice. A research group of theirs is building "friendly" worms.

By definition a standard symbiotic relationship imparts benefits on both groups. It's arguable that existing worms/botnets could be considered symbiotic worms, even "friendly", in a sense. When you click on that Valentine's day attachment, and Storm pawns your machine, one of the first changes it makes, after establishing a command and control channel, is to somewhat harden the operating system so that other worms/viruses/trojans cannot infect the new captive. This is comparable to what Microsoft's friendly worms are going to do. Storm continues from there to look for more nodes and send out spam. If Microsoft is going to label their home-brewed malware a "worm", they have some intrusive (but not that unexpected) company. Let's hope they don't decide to spin some "friendly" mass-mailings into their worms as well.

The proposed Microsoft worms would only be beneficial for the worms in the sense that the hosts allow the worms to distribute themselves. Do the worms receive other benefits from being infected by the hosts? This could be like a reverse parasite, the worm infects and the host is parasitic on the worm (what is the standard biology term for this?). Other then the partial lack of symbiosis, what distinguishes our "friendly" worm from a Storm/Mega-D infection? (1) It's not sending out Spam/performing auxiliary illegal functions (2) It's distributor is known (3) It deactivates itself? (4) It's "friendly". Point (2) will give victims legal recourse and establish an amount of responsibility with the distributor, Microsoft, this appears to be a good thing.

I genuinely hope (3) is true, this seems like the most important aspect. But, when will it deactivate? After repairing the infected system? After the previous and infecting an additional N neighbors? And, how will it deactivate and how complete will this deactivation be? On a theoretical level, it seems like you should run into some analog of the halting problem here. Virus infects machine, repairs machine, uses control (a) to delete self, control (b) to delete control mechanism (a), mechanism (c) to delete mechanism (b)... we all know were this leads, no where. What am I overlooking in this analysis? Wouldn't a control channel be needed to delete any preexisting control channels? A potential solution would be to send a message to Windows Update's servers after cleaning the machine and to have the server remove the mechanism, or it seems like some one-way function could be used as well. Contacting a Windows server is very similar to a command and control channel. (Although it's likely hopeless) let's hope no one subverts this or related channels.

With respect to (4), "Let Us Just Pour Some Oil Down This Slide and Balance Our Cluster Bombs On Top Of It."

This could go very bad very quickly for whomever performs the researching, let alone for Microsoft, a company with such a history of vulnerability-less code. Is the lack of "Windows Genuine Update" worthy of repairing? There could be a flaw in the old version of Windows Update rationalizing this view. The worm infects the victim, updates the system software, and after the user reboots they're locked out because they do not have a genuine version of Windows, regardless of whether they pirated Windows or the machine they purchased came with a pirated version unbeknownst to them. Many variations on this theme are imaginable.

Slightly more insidiously, a patch is applied to Internet Explorer closing all previous vulnerabilities and opening a new vulnerability. Has Microsoft (or any company) consistently released patches that have not created new vulnerabilities? They have not, and it's not because they're not extremely skilled programmers, it's because that is usually not a software development requirement or possibility. Patch away if the patch is going to be accompanied with a proof theoretic verifier of the patched code that (by somehow avoiding all of the paradoxes Turing, Church, and others have pointed out) proves the invulnerability of the code, this is actually more realistic than it sounds. Until then, this could easily do much more harm than good, even if performed in the friendliest of manners.

Furthermore, the potential for subversion of the worm or control channels seems intolerably significant. Any of the nearly daily (hourly?) zero-day Microsoft exploits do enough damage as it is. Imagine what would be possible with a zero-day exploit that wormed it's way through. Imagine a modification to the worm making all Windows machines appear as though they need to be patched. Reverse engineering Microsoft code is trivial compared to reversing Skype's code, and both projects have already been done and are actively being enhanced. The way in which this problem is addressed will need to be brilliant or we'll see some brilliant attackers flourish.

What are the legal implications of releasing this worm or of even pursuing research in this direction? The courts have never been to knowledgeable in the area of data security and seem to err on the side of, "if it sounds possibly not good it's horribly bad." Recall the fines given for non-malicious use of an open public WiFi networks. Whether released by Microsoft, or the Russian Business Network (RBN), a worm is a worm like sarin is sarin. If Pfizer were researching sarin to develop a variant of the gas that would repair paraplegics I'm pretty sure that would be illegal (so maybe the previous is an absurd stretch). When Iran researches nuclear weapons for "peaceful" (sounds close to "friendly") use they receive attempted sanctions and threats of invasion (from the more irrational congressmen). Does it behoove the EFF to do the equivalent? In a metaphorical way I do think so.

U.S. laws likely do not allow a computer worm to be released into the wild by a private company, certainly not by an individual. Simply performing the research is arguably illegal as well. Looking abroad, a release is certainly illegal in Germany, and likely the research as well. KisMAC moved their servers to Switzerland after Germany ratified their new cyber security laws. I'd suspect the U.K. and Australia would be rather unfriendly to these "friendly" activities as well.

Would there be legal recourse for those infected with the worm? If it's egg patching (or simply infecting) your system it will require bandwidth. In many (most?) European countries internet connections are paid for on a bandwidth basis. Microsoft is then literally stealing a commodity that you have paid for when they infect you with their worms. Shouldn't they be required to remunerate you? The virus will also use other commodities, such as disk space and processor cycles, and will through these use electricity as well as ware down your equipment (to some degree). This seems to clearly fall under unauthorized use, although it is moving into a new frontier for these types of arguments and lawsuits (I cannot think of any related precedents?).

There's always the possibility of an accident. Does Microsoft expect to develop a worm with no impact on performance/other applications/other services? They do not know every line of code that has ever been written for their products so this will certainly be a problem. If they inadvertently crash another application, maybe a load balancer, they could bring down a server or a cluster. What if it belongs to a Broker/Dealer and financial records are lost? I could image damage in the USD billions being caused, or worse, it is a worm. And, whose pockets go deeper? Most corporations would love to sue Microsoft (ignoring the formidable legal defense team for a moment), the European Union and U.S. Federal/State governments certainly seemed to enjoy it, to some extent.

This would be a service/product to be released once P == NP, "black hats" have no profit incentive, computer security laws become internationally well defined, version control is perfected and "friend" is no longer a transient term. Unfortunately, I only see one of these things ever becoming true (you know which one I'm thinking of). Apparently, Microsoft want us to go into a future in which malware is our friend.

Environmental Digitalism

noreply@blogger.com (lubell-doughtie) — Fri, 25 Jan 2008 02:47:00 +0000

Have you ever thrown out a jewel case? Or a CD, video tape, cassette tape, DVD, DVD case, the plastic packaging your DVDs and CDs come in, that annoying security sticker, or anything from AOL? Do you think anyone else has? Somewhere there must be a measure of what percentage of trash comes from package materials used for CDs, DVDs, and all the expendable media we consume. An easy way to start would be to see how much is produced. Eventually, all of that will likely be in the trash or in a recycling plant using fuel and the expensive chemical compounds that facilitate the recycling process. (I know nothing about how recycling works other than it is not a form of alchemy.)

Wouldn't it be amazing if there was some way that we could eliminate every single piece of waste that is produced by all of our consumable media?

The earth is in luck (the earth's luck seems to have been in short supply during the past couple centuries). Now instead of buying that CD or DVD with liner notes and a bar code you can eliminate your share of that form of pollution. Boot up your personal computer (which is of course also filled with all kinds of toxic chemicals) and download your media from one of the many online retailers.

Imagine if instead of eventually throwing out your jewel cases you never bought them in the first place? Imagine if you walked into Virgin Music and instead of seeing rows upon rows of plastic cases you saw interactive screens that you could browse for enticing music and then, after paying a nominal fee that primarily went directly to the artist, the album or track was transferred and licensed to play on your personal music playing device?

"I recycle all my plastic so it does not matter if I buy some jewel cases. When I get rid of it I'll recycle it and the liner notes as well."

It is more efficient to not produce the plastic in the first place, or use it to produce something more needed like tubing for irrigation. Additionally, recycling does not equate with perpetual motion: it is not resource consumption free. Similarly to fusion power there are circumstances when recycling is not efficient. (I don't know what these circumstances are but they are certainly less likely to occur than the circumstances in which fusion power is efficient, which is currently all circumstances.)

"But the liner notes! They have lyrics and pictures and biographies, plus it supports the artist."

Purchase a poster or a book about the artist with the extra money you save on not paying for plastic. Purchase something directly from the artist so they can see a nice one hundred percent of your appreciation and not the normal nine basis points. (I actually have no idea how much royalties are beyond that they are low.)

Consumers have moved on. Retailers must follow suit eventually.

The Sleekest Botnet

noreply@blogger.com (lubell-doughtie) — Thu, 24 Jan 2008 02:39:00 +0000

The Apple store is always impressive. It's not the elegant use of glass, especially glass stair cases, nor the surfeit of outwardly cheerful employees sporting classy badges. It's the fifty or so top of the line Macs that anyone can walk up to and start using.

Apple has turned into the Barnes & Noble of computer (or electronics) stores. When Barnes & Noble first appeared they were able to grab such a significant piece of market share because you could wander in, read a book, and leave without buying anything. All the local bookstores were upset since they couldn't compete with the market of scale Barnes & Noble had created. (Now copy cats, like Borders, are starting to eat into Barnes & Nobles' market share.)

Why try to clandestinely check your email at Office Depot when you can roll into Apple and spend 3 hours "friending" people? Perhaps you'll even end up buying that 29.99 pair of headphones that you could purchase for 6.99 at a corner store, at the least you'll keep the store filled.

You could also use these publicly accessibly computers to do many other things. The adage, "physical access equals game over" is still true. You could plug in a thumb drive and at the very least get the box. Maybe you could even get the LAN, and if not, just stroll over to another shiny 2ghz box and that's that. Maybe they do some hardening but I know Disk Utility works fine on the Macs (erase free space, partition, format). Being optimistic, suppose thumb drive machine access leads to owning the LAN. So that's one store, can you hop through the internal network and get another store? If not take a quick walk, subway, or taxi ride and now you have two LANs. Is this possible (yes)? Is this hard (probably not)? Failing a thumb drive, you have internet access on the box, why not just download?

Has this not happened yet? If it has what has the exploiter gained? Very simplistically you could walk in and go to a Mac, download and install a stealthy key logger (hardware or software), then walk out and see what happens. I've seen lots of people checking email accounts (I've checked my low security email account) on an Apple store Mac. Have you ever seen someone checking a bank account? Even if not, email account access gives an extremely large amount of information, identity theft anyone?

If this has not happened yet (and I doubt it) it will happen soon and someone, then many people, will learn why public terminals are dangerous.

Update: I've noticed some Macs that are more locked down, I think they have parental controls on. Still, it's a closed source operating system, who can't break mac parental controls on those?

The Automated Future

noreply@blogger.com (lubell-doughtie) — Wed, 23 Jan 2008 02:37:00 +0000

Normally I stay away from stores that vend a majority of poisonous substances however, recently, while in a Stop & Shop, I was given a vision of the future. A future were human interaction is the exception. It is something that occurs a couple times a week, something you might look forward to or something you might dread, in any case it is a rarity. Relish the moments you spend waiting for the incredibly slow cashier to change your twenty; the moments you spend chatting with them about the upcoming elections. Those opportunities are quickly disappearing.

The local Stop & Shop installed some (three or four?) self-checkout systems. These MRI-reminiscent devices do exactly what the name connotes. Scan your items, choose your payment method, enter your card info or insert your cash, and walk away.

I consider the honor system a wonderful form of enforcement. However, it is also an unrealistic form of enforcement in the current society of the United States. In the store that I was in the lines which used self-checkout were solely the 12 item or fewer express lines. I don't believe there existed any formal verification that the items you placed in your bag(s) were items that you scanned and payed for. There was an employee who bagged my item (not that my single 1x1x4 in. item needed a bag) but she had no idea whether I paid or not and I doubt she cared given the attitudes of the majority of Stop & Shop employees (that is of course no fault of hers, it may be difficult to care about such a job). What prevents someone from weighing and paying for only an ounce of the carrots out of the 5 ounces of carrots they purchase? If stopped, there would be a reasonable amount of plausible deniability, "Oh, it works that way? I'm so new to these machines!" Or, what prevents someone from swiping a stick of gum and then swiping a (insert something more expensive than a stick of gum) while holding their hand over the bar code? Still, lots of deniability, "Really? That didn't scan? I swiped it and everything?" Not that there would necessarily be anyone to stop you on your way out, although mind the innumerable security cameras.

Maybe that is the future we have to look forward to. After underpaying for a couple of ounces of rice, and not the pound you took, an automated surveillance system uses the fact that the rest of the self-checkout system also acts as an unbeknownst scale and notices a discrepancy between the weight of your rice on the scale and the weight in the bag. As you push against the door to leave the store you realize it's locked so turn around to push against the door behind you and realize your caught in the double door exit. You wait in between the doors for about 15 seconds before seeing a couple Stop & Shop security guards heading your way. It is the first time you've ever seen a human employee of Stop & Shop.